1879227 - Improper Protocol Resolution In Fetch() API

Status: RESOLVED INVALID

(Core :: Networking, defect)

Description

[Aviv Keller]

Steps to reproduce:

1. Navigate to a HTTPS webpage, such as "https://example.com".

2. Open the browser's Developer Tools interface.

3. Within the Developer Tools console, execute the command fetch("http:web.site"), where "web.site" represents any desired website accessible via HTTP.

Note the successful attempt to retrieve the specified HTTP webpage.

4. Subsequently, execute the command fetch("https:web.site") within the Developer Tools console, with "web.site" representing any desired website accessible via HTTPS.
   Observe the unexpected behavior, which may entail an attempt to fetch https://example.com/web.site rather than the intended target.

(This also works for the vise-versa)

Actual results:

When you execute fetch("https:web.site"), the browser interprets the URL incorrectly due to the missing forward slashes (//) after https:. As a result, the browser appends the provided URL fragment (web.site) to the current base URL (https://example.com/) instead of treating it as an absolute URL. This behavior is due to how browsers handle incomplete or malformed URLs when using the fetch API.

For instance, if the current page is https://example.com/, executing fetch("https:web.site") would result in the browser attempting to fetch https://example.com/web.site instead of the intended target https://web.site.

Expected results:

The browser should have correctly interpreted the provided URL https:web.site as an absolute URL, just like it does for properly formatted URLs such as https://web.site. The absence of forward slashes (//) after the scheme (https:) should not lead to the URL being concatenated with the base URL of the current webpage.

Thus, executing fetch("https://web.site") within the Developer Tools console should have resulted in the browser attempting to fetch the content from https://web.site, as expected, without any unexpected behavior or URL concatenation. Similarly, executing fetch("http://web.site") should fetch the content from http://web.site without any issues related to URL interpretation.

Comment 1

[Aviv Keller]

I have identified this issue on almost every major browser (and I have reported it to the Chromium team for them to resolve on their browsers)

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Security' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Aviv Keller]

after further testing; this issue applies not only to the Fetch API, but every part of the browser.
I will be creating a PoC website later today.

Comment 4

[Tom Schuster]

This doesn't sound like a bug to me, but the behavior expected by the WhatWG URL specification. You can see the same result https://example.org/web.site with https://jsdom.github.io/whatwg-url/#url=aHR0cHM6d2ViLnNpdGU=&base=aHR0cHM6Ly9leGFtcGxlLm9yZw==.

Comment 5

[Aviv Keller]

Okay, but I still think it’s an issue, maybe with WhatWG. I’ll ask them and find out. Thank you

Comment 6

[Valentin Gosu [:valentin]]

I agree. The URL is parsed according to the URL standard

This specific case would be a validation error, but either way the parsing is correct and all browsers agree on this behaviour.

BASE	INPUT	Result	Link
http://example.com	http:web.site	http://example.com/web.site	Link
http://example.com	https:web.site	https://web.site	Link
https://example.com	http:web.site	http://web.site	Link
https://example.com	https:web.site	https://example.com/web.site	Link

Comment 7

[Aviv Keller]

Thank you. I have filed an issue with WhatWG to obtain more information: https://github.com/whatwg/url/issues/822

Comment 8

[Aviv Keller]

From WhatWG on https://github.com/whatwg/url/issues/822:

This is a very well written issue report, however, I'm afraid that the request goes counter to one of our goals: url.spec.whatwg.org/#goals. In particular getting alignment across implementations. This change would likely not be web compatible and lead to broken experiences for end users.

(Apologies if browsers match what you propose and you just forgot to write that down. I'm pretty sure they don't, but I didn't double check.)

I was wondering if Mozilla/Firefox would approve of this change, or if it would break the browser as we know it?

Comment 9

[Valentin Gosu [:valentin]]

Since all browsers currently agree on this behaviour, there is little incentive to change it.
One certain thing is that there's a risk of regressions when making this change, so the browsers with the higher market share would have to change it first.
For now this is one of those unfortunate quirks of the web. 🤪

Comment 10

[Aviv Keller]

Got it, thanks! The web is wonderful place!