1495513 - Perma TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/get_current_url/get.py | expected OK

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P2)

Description

[Treeherder Bug Filer]

Filed by: hskupin [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=201955384&repo=try

https://queue.taskcluster.net/v1/task/ApeMKb9fQNq8SBBnGalm0Q/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is actually a hang in "Webdriver:Navigate" when loading `file:///`:

> 14:17:55     INFO - PID 5280 | 1538057875037	Marionette	TRACE	0 -> [0,78,"WebDriver:Navigate",{"url":"file:///"}]
> 14:17:55     INFO - PID 5280 | 1538057875045	Marionette	DEBUG	[6442450945] Received DOM event beforeunload for about:blank
> 14:17:55     INFO - PID 5280 | 1538057875060	Marionette	DEBUG	[6442450945] Received DOM event pagehide for about:blank
> 14:17:55     INFO - PID 5280 | 1538057875061	Marionette	DEBUG	[6442450945] Received DOM event unload for about:blank
> 14:17:55     INFO - PID 5280 | 1538057875062	Marionette	DEBUG	[6442450945] Received observer notification outer-window-destroyed
> 14:17:55     INFO - PID 5280 | 1538057875360	Marionette	DEBUG	[8589934593] Frame script loaded
> 14:17:55     INFO - PID 5280 | 1538057875366	Marionette	DEBUG	[8589934593] Frame script registered
> 14:17:55     INFO - PID 5280 | 1538057875411	Marionette	DEBUG	[8589934593] Check readyState uninitialized for about:blank
[..]
> 14:19:09     INFO - TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/get_current_url/get.py | expected OK

I will mark the test as disabled via my patch on bug 1370636.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I think the problematic section in the code is here:
https://dxr.mozilla.org/mozilla-central/rev/dcba2a476ccfb77f859b3614d2c77ed3cc06a225/testing/marionette/listener.js#148-165

Maybe it is a race between checking for the `document.readyState` and adding the event listeners for `DOMContentLoaded` and `pageshow`. 

But what makes me wonder is why this hang only happens for the wdspec tests, but not the Marionette unit tests in test_navigation.py, which also contains a couple of page loads for `file:///`.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This actually seems to be a problem with Firefox on Windows. Even in a regular profile I'm not able to load `file:///`. But I can load `file:///C:/`. So I assume that if no drive letter has been specified, loading this URL fails.

Bob, can you shed some light on that? AFAIR you worked on loading such URLs not too long ago. Thanks.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I'm fairly sure that the expected result should be an error page similar when trying to open `file:///c:/` under Linux or MacOs.

Comment 5

[Intermittent Failures Robot]

1 failures in 2584 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495513&startday=2018-10-22&endday=2018-10-28&tree=all

Comment 6

[Bob Owen (:bobowen)]

(In reply to Henrik Skupin (:whimboo) [away 12/05-12/16] from comment #3)
> This actually seems to be a problem with Firefox on Windows. Even in a
> regular profile I'm not able to load `file:///`. But I can load
> `file:///C:/`. So I assume that if no drive letter has been specified,
> loading this URL fails.
> 
> Bob, can you shed some light on that? AFAIR you worked on loading such URLs
> not too long ago. Thanks.

Sorry about the delay here, I've been a bit slow on needinfos recently, but I must have missed this one or I'd have answered earlier.
I worked on the process that they load in, but not the actual loading itself.
I guess it comes under networking ... is this a regression?

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Bob Owen (:bobowen) (PTO until 3rd Jan) from comment #6)
> I guess it comes under networking ... is this a regression?

I cannot say if that is a regression or not because we didn't run the test on Windows before. Which kind of extra logs would be helpful to investigate this problem?

Comment 8

[Bob Owen (:bobowen)]

(In reply to Henrik Skupin (:whimboo) [away 12/21-01/01] from comment #7)
> (In reply to Bob Owen (:bobowen) (PTO until 3rd Jan) from comment #6)
> > I guess it comes under networking ... is this a regression?
> 
> I cannot say if that is a regression or not because we didn't run the test
> on Windows before. Which kind of extra logs would be helpful to investigate
> this problem?

I've just tried mozregression to load firefox 31.0a1 on Windows and file:/// does not work.
It appears to just do nothing.
So, it looks like this needs fixing in the test.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Bob Owen (:bobowen) (PTO until 3rd Jan) from comment #8)
> I've just tried mozregression to load firefox 31.0a1 on Windows and file:///
> does not work.
> It appears to just do nothing.

This won't work given that Marionette at this time wasn't that feature packed and only starting from Firefox 56 can load file:// URLs. I implemented that via bug 1332122.

> So, it looks like this needs fixing in the test.

No, the test is fine and works in other browsers. So I will ask my question again, which kind of logs would be helpful for you so we can further investigate it.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Henrik Skupin (:whimboo) [away 12/21-01/01] from comment #3)
> This actually seems to be a problem with Firefox on Windows. Even in a
> regular profile I'm not able to load `file:///`. But I can load
> `file:///C:/`. So I assume that if no drive letter has been specified,
> loading this URL fails.

Actually.... I missed that I already found the reason why it is failing. So maybe you can just verify that this behavior is correct and that we cannot assume that loading `file:///` on Windows (and vice versa) will work.

But I still wonder why it just hangs. So I assume there might still be a bug in Firefox?

Comment 11

[Bob Owen (:bobowen)]

(In reply to Henrik Skupin (:whimboo) [away 12/21-01/01] from comment #9)
> (In reply to Bob Owen (:bobowen) (PTO until 3rd Jan) from comment #8)
> > I've just tried mozregression to load firefox 31.0a1 on Windows and file:///
> > does not work.
> > It appears to just do nothing.
> 
> This won't work given that Marionette at this time wasn't that feature
> packed and only starting from Firefox 56 can load file:// URLs. I
> implemented that via bug 1332122.

I wasn't using Marionette, just Firefox.

> > So, it looks like this needs fixing in the test.
> 
> No, the test is fine and works in other browsers. So I will ask my question
> again, which kind of logs would be helpful for you so we can further
> investigate it.

file:/// doesn't appear to work in IE or Chrome on Windows either.

> Actually.... I missed that I already found the reason why it is failing. So
> maybe you can just verify that this behavior is correct and that we cannot
> assume that loading `file:///` on Windows (and vice versa) will work.
> 
> But I still wonder why it just hangs. So I assume there might still be a bug
> in Firefox?

Possibly, the other browsers give some sort of error which makes more sense, presumably the test is reacting to that error.
However, like I said in comment 6, I don't think I'm the correct person to look at file:// URL loading issues, you probably want someone from the network team.

Comment 12

[James Graham [:jgraham]]

I thought the hang with file:/// was a known thing because file:/// urls are run in a seperate process and we don't handle the procesess change correctly in marionette?

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to James Graham [:jgraham] from comment #12)
> I thought the hang with file:/// was a known thing because file:/// urls are
> run in a seperate process and we don't handle the procesess change correctly
> in marionette?

No, process changes are working quite well for some time with Marionette, otherwise we wouldn't be able to load a lot of URLs those days due to multiple content processes. This is an underlying Firefox bug.

(In reply to Bob Owen (:bobowen) (PTO until 3rd Jan) from comment #11)
> > Actually.... I missed that I already found the reason why it is failing. So
> > maybe you can just verify that this behavior is correct and that we cannot
> > assume that loading `file:///` on Windows (and vice versa) will work.
> > 
> > But I still wonder why it just hangs. So I assume there might still be a bug
> > in Firefox?
> 
> Possibly, the other browsers give some sort of error which makes more sense,
> presumably the test is reacting to that error.
> However, like I said in comment 6, I don't think I'm the correct person to
> look at file:// URL loading issues, you probably want someone from the
> network team.

Maybe Patrick can help or redirect to the right person here. Just to summary the problem... while file:/// is a valid URL on *nix systems, it cannot be loaded on Windows. Usually it should result in a not found error, but currently Firefox on Windows loads this page forever. Do you have an idea why this happens? Thanks

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It looks like Patrick isn't with us anymore. Valentin, maybe you could help out, or maybe know the person who has knowledge in that area? Please see my last comment. Thanks!

Comment 15

[Valentin Gosu [:valentin] (he/him)]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #14)

It looks like Patrick isn't with us anymore. Valentin, maybe you could help out, or maybe know the person who has knowledge in that area? Please see my last comment. Thanks!

Indeed, I would expect that loading file:/// would fail on windows, but if it just keeps loading forever, that seems like a bug. I'll take a look at it.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thanks a lot!

Comment 17

[Intermittent Failures Robot]

1 failures in 2383 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495513&startday=2019-01-21&endday=2019-01-27&tree=all

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Valentin, do you know when you will be able to have a look at this bug? Maybe we should file a new one under networking?

Comment 19

[Valentin Gosu [:valentin] (he/him)]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #18)

Valentin, do you know when you will be able to have a look at this bug? Maybe we should file a new one under networking?

Hi Henrik, I was away from my windows machine and I have a high priority project I have to take care of, but I should be able to get to it next Monday, if that's OK.
Otherwise I can try a building in a VM, if it's urgent.

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+2]]

No, it's not urgent. I only wanted to check if you already had a chance to take a look. Thanks.

Comment 21

[Valentin Gosu [:valentin] (he/him)]

The problem seems to be coming from here:

[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIWebNavigation.loadURI]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource://gre/actors/WebNavigationChild.jsm :: loadURI/< :: line 126"  data: no]
    loadURI resource://gre/actors/WebNavigationChild.jsm:126
    _wrapURIChangeCall resource://gre/actors/WebNavigationChild.jsm:63
    loadURI resource://gre/actors/WebNavigationChild.jsm:125
    receiveMessage resource://gre/actors/WebNavigationChild.jsm:43
    receiveMessage resource://gre/modules/ActorManagerChild.jsm:160

_wrapURIChangeCall in WebNavigationChild.jsm doesn't propagate the errors it gets from loadURI/LoadURIFromScript, so it just stalls. When I removed the try {} block _wrapURIChangeCall, the load failed immediately. Also, in non-e10s, loading file:/// fails immediately.

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Mike, maybe you could also have a look at Valentin's reply? Maybe Kris is not around those days. Thanks!

Comment 23

[Mike Conley (:mconley) (:⚙️)]

_wrapURIChangeCall was added in bug 1241085, but even before then we had a try/catch around the loadURIWithOptions call:

https://hg.mozilla.org/mozreview/gecko/rev/dea0a728eec67a7c78ac57ce34e83f7d554136fa#l1.58

that try/catch was added in bug 798249.

Can you tell me more about this bug? Does it only manifest for Marionette, or is this something that could impact our users? If so, what are the STR?

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #23)

Can you tell me more about this bug? Does it only manifest for Marionette, or is this something that could impact our users? If so, what are the STR?

Just open Firefox on Windows and try to load file:/// via the location bar. The spinner will run forever. So yes, it impacts users.

Comment 25

[Mike Conley (:mconley) (:⚙️)]

STR:

1. Load file:///INVALID in the URL bar and press enter

ER:

The file URI is invalid, and should result in some kind of error message in the tab.

AR:

The tab blanks out, and seems to load forever (the tab throbber throbs).

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is a test failure, so please don't move to a different component, or dupe it. Thanks for investigating, Mike.

Comment 28

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Maybe we should just use different file URIs for Windows and Linux/MacOS, so we can make sure that we will always get correct content displayed.

Comment 29

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So we can basically use server_config["doc_root"] and prefix it with file:// to hopefully have it platform independent.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e03a210ac9d5384e131f543edd8ad5d1e5c4bb9e

Comment 30

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Bug 1495513 - [wdspec] Add tests for file protocol. r?ato

Comment 31

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/be197c4fadf0
[wdspec] Add tests for file protocol. r=ato

Comment 32

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/be197c4fadf0

Comment 33

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/15547 for changes under testing/web-platform/tests

Comment 34

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged

Comment 35

[BMO Automation]

Moving bug to Remote Protocol::Marionette component per bug 1815831.