1797376 - browsingData.remove with hostnames parameter behaves inconsistently/broken for ports and IPv6 addresses

Status: NEW

(WebExtensions :: General, defect, P3)

Description

[Rob Wu [:robwu]]

The browsingData.remove* methods have a "hostnames" parameter. The name and documentation suggests that a host name should be passed. The browsing_data.json schema refers to the hostname format, whose implementation only accepts the input if it is the same as new URL(...).host.

While often equal, a "host" is NOT the same as "host name". The two main differences are:

1. "host" may include a port ("host" = "hostname:port")
2. IPv6 addresses are wrapped in brackets in the public URL and extension APIs. Most of Firefox's internals expect IPv6 addresses without brackets.

To add to the confusion, "host name" and "domain" are often used interchangeably. But when "domain" is used, ambiguity over what a "domain" matches occurs: exact equality vs subdomain overlap vs superdomain overlap.

With the above definitions of "host name", "host (with port)" and "domain" in mind, let's review how the browsingData.remove API implementation handlers the hostnames parameter:

• clear cache by hostname
  • NOTE: treats hostname as host + port pair. The underlying implementation ends up concatenating the "host" with "http://" and "https://" to delete data (e.g. NetworkCacheCleaner; ImageCacheCleaner and CSSCacheCleaner are similar).
• remove cookie by hostname
  • cookie.host is without port. IPv6 should be without brackets.
• clear serviceWorkers
  • NOTE: interpreted as a domain.
• clear localStorage / indexedDB
  • NOTE: this compares with principal.hostPort.
  • localStorage by hostname when LSNG is off (unsupported; it's on by default)
    • NOTE: Interpreted as a "domain". It should not have a port, and IPv6 addresses should not have brackets.
    • NOTE: This behavior of unexpectedly removing subdomain data was observed in bug 1595431.

As you can see, all three meanings of "host"/"host name"/"domain" are used interchangeably.

Note for completeness: clearing by "hostname" is not implemented for the following types:

• "downloads" - bug 1643917
• passwords
• formData
• history

DataTypeSet lists more:

• fileSystems - not present in the schema, documented as unsupported.
• pluginData - no-op in Firefox, so lack of support doesn't matter.
• serverBoundCertificates - although existent in the schema, there is no implementation (and also documented as unsupported in the BCD).

Work-arounds for extension developers

Extensions can resolve the ambiguity of "host name" vs "host (without port)", by adding or removing the port, and calling the API twice, with and without port when relevant.

Due to the bracket notation, browsingData.remove will fail at deleting cookies for IPv6 addresses. The work-around is to use the cookies API instead of the browsingData API.

Next steps

• Rename or remove the inaccurately-named hostname format. If removed, "format": "hostname" can be removed from the browsing_data.json schema in favor of validation in the ext-browsingData implementation itself.
• Decide on whether to update the implementation to match the documented behavior and API name.
  • If we treat the input as documented, i.e. drop support for the port parameter, then extensions need a different way to remove data for a specific "host + port".
  • "host + port" is still ambiguous, because data is typically keyed by an origin, which includes a scheme. I strongly recommend the implementation of origins (bug 1632796), which is unambiguous.
• If support for the current behavior is kept (host + port), then a way of removing by IPv6 addresses is needed.

Comment 1

[Giulio B]

We are hitting this in our extension webcat#137. Even when accounting for the host vs. host:port confusion, the hostnames filter still clears effectively nothing for cache, only entries whose OriginAttributes happen to be empty (extension fetch requests, some internal Firefox requests). Anything cached from a normal tab, and that thus has partitionKey is skipped.

To test, visit https://example.com/ in a normal tab, then from an extension background page call browser.browsingData.remove({hostnames:['example.com']}, {cache:true}).

In about:cache?storage=disk the https://example.com/ entry (with O^partitionKey=(https,example.com)) survives.

Otherwise, first perform a fetch via the same extension background page to https://example.com, then retry the remove(). For the fetch cache entry, it should succeed.

In the same file where the cleaning primitive is defined, it seems like maybe there is a more suited one, that seems partititon aware: deleteBySite. Maybe it's worth migrating this method to invoke that one instead?

On our side, it would be really nice to get this fixed, because we need to selectively clear the cache to prevent some type of attacks. If there's clear next steps forward for which implementation decision to make, we'd also be happy to contribute ourselves.