1847736 - Create url pattern helper for bidi network intercept matching

Status: RESOLVED FIXED

(Remote Protocol :: WebDriver BiDi, task, P1)

Description

[Julian Descottes [:jdescottes]]

An important part of BiDi network interception is to implement a simplified URL pattern matching logic as described in https://pr-preview.s3.amazonaws.com/w3c/webdriver-bidi/pull/429.html#parse-url-pattern

Extracted from Bug 1826192.

Comment 1

[Julian Descottes [:jdescottes]]

Attachment: Bug 1847736 - [remote] Create URLPattern module to match simple url patterns

Comment 2

[Julian Descottes [:jdescottes]]

For what it's worth, trying out the URLPattern implemented in Chrome, they do not differentiate between empty query string and no query string. I believe it's rather a limitation from their implementation rather than something valid according to the spec, but it's worth noting.

Comment 3

[Julian Descottes [:jdescottes]]

Ran the current matchURLPattern tests against chrome's URLPattern implementation for comparison.

Beyond known edge cases already spotted while writing the test and mentioned on the spec PR, the tldr is:

• chrome urlpattern does not support ipv6
• chrome urlpattern does not add a leading "/" to pathname properties
• chrome urlpattern does not support user/password
• chrome urlpattern handles schemes with different cases ... weirdly...
• chrome urlpattern considers the hash for matching URLs
• chrome urlpattern does not differentiate between no query or empty query

From those I think the path, hash and query are the most interesting to investigate to see if they're correct or not. The rest feels like implementation limitation/bug.

Detailed failures below

Failures

(3) Discussed at https://github.com/w3c/webdriver-bidi/pull/429#discussion_r1284443254
Introduce hasPathname flag, when pathname is not provided, we still default to "/" because of the http scheme.

• url "http://example.com/path" should not match pattern "{"hostname":"example.com"}"
• url "http://example.com/a" should not match pattern "{"protocol":"http"}"
• url "http://whatever.com/path?search#ref" should not match pattern "{"protocol":"http"}"

(1) Discussed at https://github.com/w3c/webdriver-bidi/pull/429#discussion_r1282752003
Similar to above

• url "http://example.com:1234" should not match pattern "{"protocol":"http"}"

(1) Discussed at https://github.com/w3c/webdriver-bidi/pull/429#discussion_r1284394991
Match non-null search if search is null in the pattern

• url "http://example.com/?abc" should match pattern "http://example.com"

(1) Chrome URLPattern seems to handle schemes with a different case in a weird way:
pattern -> url -> result
(if the pattern uses http, it seems to match as in our implementation)
http://example.com:1234 -> HTTP://example.com:1234 -> matches
http://example.com:80 -> HTTP://example.com:80 -> matches
http://example.com:80 -> HTTP://example.com -> matches
http://example.com -> HTTP://example.com:80 -> matches
http://example.com -> HTTP://example.com -> matches

(if the pattern uses HTTP however, it never matches if the pattern also
defines the default port, but it works with non default ports!? Feels like a
bug on Chrome side)
HTTP://example.com:1234 -> http://example.com:1234 -> matches
HTTP://example.com:80 -> http://example.com:80 -> fails
HTTP://example.com -> http://example.com:80 -> matches
HTTP://example.com:80 -> http://example.com -> fails
HTTP://example.com -> http://example.com -> matches

• url "http://example.com" should match pattern "HTTP://example.com:80"

(1) Chrome URLPattern does not add a leading "/" to URLPatternPattern pathname
In our current spec: If pathname does not start with U+002F (/), then append "/" to pattern.
Chrome's approach seems sensible for things such as new URLPattern({"pathname":"path"}).test("uri:path")
Basically in Chrome, a pattern with a pathname which does not start with "/" will not match any special-scheme URL. And something like new URLPattern({protocol: "http", "pathname":"path"}) will most likely not match any URL ever.

• url "http://example.com/path" should match pattern "{"pathname":"path"}"

(1) Does not seem to support user/password at all?
The test below failed, but even matching "http://user:[e-mail]" with
itself failed.

• url "http://user:[e-mail]" should match pattern "http://example.com"

(3) Chrome URLPattern considers the hash
Should we do the same?

• url "http://example.com/#some-hash" should match pattern "http://example.com"
• url "http://example.com/search?param#ref" should match pattern "http://example.com/search?param"
• url "http://example.com/search?param" should match pattern "http://example.com/search?param#ref"

(2) Chrome URLPattern does not differentiate between empty search and no search
Same limitation as our current new URL.

• url "http://example.com/" should not match pattern "{"search":""}"
• url "http://example.com/emptysearch" should not match pattern "http://example.com/emptysearch?"

(2) Chrome URLPattern does not support creating patterns which use ipv6 hostnames, so I could not run the tests based on that.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c4c0eea3b248
[remote] Create URLPattern module to match simple url patterns r=webdriver-reviewers,whimboo

Comment 5

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/c4c0eea3b248