1601816 - dweb, dat, ipfs, ipns, ssb, wtp, ssh are incorrectly treated as WHATWG URL “special scheme”

Status: RESOLVED FIXED

(Core :: Networking, defect, P3)

Description

[ntninja]

Currently, a dweb:-URI parsed using JavaScript new URL returns:

>>>  new URL("dweb:/ipns/blabl/")
URL { href: "dweb://ipns/blabl/", origin: "dweb://ipns", protocol: "dweb:", username: "", password: "", host: "ipns", hostname: "ipns", port: "", pathname: "/blabl/", search: "" }

Fixing up the dweb: path section into section by segmenting it into a hostname and a path component and assigning an obviously wrong origin value to the result (the string “ipns” in this case is a well-known discovery type verb and in no way unique to any specific site).

AFAIK, this kind of processing is the result of the exact algorithm prescribed by the WHATWG URL standard for “special schemes”.

Since the exact semantics for the dweb:-scheme are not known yet (“special scheme URL”-style treatment maybe being the right thing to do and maybe not…), it would be better to instead treat the dweb:-URI as similar to the mailto:-URI scheme (note the lack of assigned origin!):

>>>  new URL("dweb:/ipns/blabl/")
URL { href: "dweb:/ipns/blabl/", origin: "null", protocol: "dweb:", username: "", password: "", host: "", hostname: "", port: "", pathname: "/ipns/blabl/", search: "" }

Quick experiments also indicate that this is the default treatment for unknown schemes by Firefox anyhow.

Comment 1

[Daniel Bodea [:danibodea]]

I cannot confirm this bug since I do not know how to. I have set the component to (Core) DOM: Core and HTML because it seemed correct.
@DEV/assignee: do you need it confirmed? I'll have to ask for a test case from the reporter if the case.

Comment 2

[Frédéric Wang (:fredw)]

Just stumbled on this bug too while testing some ipfs stuff.

Special names are defined here in the spec: https://url.spec.whatwg.org/#special-scheme
The URL parser algorithm is https://url.spec.whatwg.org/#concept-basic-url-parser ; for non-special urls, it will go directly to https://url.spec.whatwg.org/#cannot-be-a-base-url-path-state after parsing the scheme, and append remaining part to the path / query / fragment.

Here is a simple test. WebKit and Chromium returns an empty host and the specified "A/B" path as pathname:

["dweb",
"dat",
"ipfs",
"ipns",
"ssb",
"wtp",
"foo",
"bitcoin",
"geo",
"im",
"irc",
"ircs",
"magnet",
"mailto",
"mms",
"news",
"nntp",
"openpgp4fpr",
"sip",
"sms",
"smsto",
"ssh",
"tel",
"urn",
"webcal",
"wtai",
"xmpp"].forEach(nonspecialscheme => {
let url = new URL(${nonspecialscheme}:A/B);
console.log(nonspecialscheme, url.host, url.pathname);
});

We have special handling for these schemes in source/netwerk/base/nsNetUtil.cpp and it seems this was added in bug 1536744 and bug 1271553.

Comment 3

[Anne (:annevk)]

Valentin, can we start trimming this list and also mark it legacy/deprecated somehow so people don't add to it?

Comment 4

[Sébastien Règne]

I think the bug URL constructor does not support unknown protocols is similar.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Bug 1603699 resolves this issue, and all schemes are now parsed according to the URL standard.