Open Bug 1596935 Opened 3 years ago Updated 2 months ago

Firefox doesn’t resolve <link rel=dns-prefetch> on HTTPS

Categories

(Core :: DOM: Networking, defect, P3)

Product:
Core
Component:
DOM: Networking
Version:
70 Branch
Type:
defect

Tracking

()

Project Flags:
Performance Impact medium

People

(Reporter: code, Unassigned)

References

Details

(Keywords: perf:pageload, Whiteboard: [necko-triaged])

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36

Steps to reproduce:

  1. Add <link rel="dns-prefetch" href="https://mozilla.org/"> to a document loaded over HTTPS

Actual results:

Nothing. The preference network.dns.disablePrefetchFromHTTPS defaults to true and blocks this from working. It works as expected on HTTP, but HTTPS is kind of the default now.

Expected results:

The DNS for mozilla.org should be resolved early. <link rel="preconnect" href="https://mozilla.org/"> works on HTTPS and it “leaks” DNS resolution just the same as rel="dns-prefetch". The policy is inconsistent. The preference for network.dns.disablePrefetchFromHTTPS should be changed to false by default.

Chrome (Android, desktop) and Safari (macOS only) supports <link rel="dns-prefetch"> on HTTPS origins by default. (Safari on iOS doesn’t support it at all.)

This was a regression, introduced by bug 1572505.
smaug, was this overseen in the review or we really wanted to disable dns-prefetch?

Flags: needinfo?(bugs)

This is not a regression.

I’ve tested versions all the way back to 30 (my PC won’t run older versions) and dns-prefetch doesn’t work in any version on HTTPS. Works fine on HTTP in all versions.

In Chrome and Safari, explicit lookups with <link rel="dns-prefetch"> works for both HTTPS and HTTP documents. In Firefox, it only works on HTTP. This bug is a request to change Firefox behavior so explicit lookups also works on HTTPS.

I believed “network.dns.disablePrefetchFromHTTPS” controlled that behavior, as setting it to TRUE makes that change happen. I’m sorry for the confusion if that setting also controls other behavior.

I presume there was a reason Chrome didn't prefetch https back then (and we followed them); what changed or what did people find out that changed the analysis? Perhaps a privacy issue?

I presume there was a reason Chrome didn't prefetch https back then

I can’t find any evidence there has ever been a test for the protocol when resolving <link rel="dns-prefetch"> in either WebKit or Chromium sources. I’ve dug through different versions of their source code all the way back to 2008. It was unintentionally broken on protocols other than HTTP in WebKit for a few months in 2010 (this was after Firefox had implemented the feature).

If I was a betting man, I’d guess that this document is the source of the confusion. It was referenced in bug #453403 (tracking the feature implementation in Firefox). The document can be read in a way that suggests that Chrome doesn’t resolve <link rel="dns-prefetch"> hints on HTTPS websites. (It doesn’t say anything about it, though.) Comments in that bug reference this behaviour. The feature implementer wrote a blog post about his work at the time. It also references this document and behavior.
https://dev.chromium.org/developers/design-documents/dns-prefetching
https://bitsup.blogspot.com/2008/11/dns-prefetching-for-firefox.html

Firefox not resolving dns-prefetch on HTTPS origins seems to just have been a misunderstanding.

Summary: Switch network.dns.disablePrefetchFromHTTPS default to false → Firefox doesn’t resolve <link rel=dns-prefetch> on HTTPS
Priority: -- → P2

This came out of bug 492196 comment 16 I suspect. It seems there were tracking concerns at the time, but I'm not sure how those are different from <link rel=stylesheet> or some such for HTTPS.

Flags: needinfo?(bzbarsky)

No, that just refactored some code. The origin of this pref and its default value is bug 453403, which is the original landing of DNS prefetch.

Bug 453403 comment 17 mentions "Do we want something akin to chromium's https thing here" but I don't see in that bug what that thing was, nor do I recall what I was talking about there; sorry, it's been 11 years. :)

The tail end of bug 453403 comment 18 does sound like at the time Chromium did not do DNS prefetch for https.

I guess all that is summarized in comment 7.

In terms of what we should do now... Martin, do you have any opinions here?

Flags: needinfo?(bzbarsky) → needinfo?(mt)
Status: UNCONFIRMED → NEW
Ever confirmed: true

How about enabling dns-prefetch and preconnect only if DoH is used?

How about enabling dns-prefetch and preconnect only if DoH is used?

Well, you can resolve DNS (and open a TCP connection and perform a TLS handshake) using <link rel="preconnect">. This is supported in Firefox on HTTP and HTTPS websites. I don’t see why <link rel="dns-prefetch"> would be any more of a privacy problem on a HTTPS page than preconnect, prefetch, preload, stylesheet, or any other link relationship that lets website load external content. <img src> would also let you resolve an arbitrary domain name from a HTTPS page.

It seems there were tracking concerns at the time […]

The (notably ten years old and possibly out of date) Chrome design document talks about disabling DNS-prefetching of domains for <a href> on hover. I can’t find anything on the web with anyone talking about privacy issues with the <link rel="dns-prefetch"> mechanism.

I'd have to do some analysis to determine what the privacy consequences of prefetching would look like. Those are not completely obvious. My intuition is that Daniel's assertion in comment 11 is probably correct, but we should do due diligence.

Flags: needinfo?(mt)

Note: knowing now (I thought it was on by default all this time) that dns-prefetch has not been used by default, it would be worth going through the code and checking if we do all security checks properly, if principals are correct, etc. before turning it on by default. Also making sure we have tests. (Maybe we do, I do not know, but there are probably people that knows)

Component: DOM: Core & HTML → DOM: Networking
Priority: P2 → P3
Whiteboard: [necko-triaged]
See Also: → 1583298

I don't have any evidence that it's related to dns-prefetch, but in Bug 1583298 I did collect some results where spend more time in DNS lookup than Chrome.

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #14)

I don't have any evidence that it's related to dns-prefetch, but in Bug 1583298 I did collect some results where spend more time in DNS lookup than Chrome.

How mush work would it be to retest, maybe just a part, with pref turn on?
Probably it would be faster to look at web sites and see if they send rel=dns-prefetch

(In reply to Dragana Damjanovic [:dragana] from comment #15)

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #14)

I don't have any evidence that it's related to dns-prefetch, but in Bug 1583298 I did collect some results where spend more time in DNS lookup than Chrome.

How mush work would it be to retest, maybe just a part, with pref turn on?
Probably it would be faster to look at web sites and see if they send rel=dns-prefetch

Sorry, I missed this question.
I would be happy to re-test this. I have a bit of a testing queue at the moment, so it will have to wait a little bit.
I didn't realize this was switched by pref (network.dns.disablePrefetchFromHTTPS I assume?)

One thing I'm a bit concerned about in local testing is that we make use of the OS's DNS cache, right?
This could mask the impact of the change.
So perhaps I should clear the OS DNS cache between pageloads?

Flags: needinfo?(dd.mozilla)

I ran two identical tests comparing baseline with network.dns.disablePrefetchFromHTTPS set to false
Each of the tested sites make use of rel="dns-prefetch" via https.
These are live site tests and in between each cold page load I also flush the Windows DNS cache via ipconfig

From eyeballing it, there may be wins on some sites, although they are not necessarily consistent run-to-run.
For the 2nd run (on the 2nd tab), I removed some of the noisier sites:
https://docs.google.com/spreadsheets/d/1adlI3M5eCf7YoNnrfdJ7aEoxPWz8t08RQFkUyaSBPkQ/edit#gid=736102674

I added additional metrics, including domainLookupTime, (which is domainLookupEnd - domainLookupStart).
I also compared against a run where I did not flush the OS dns cache between cold page loads.

https://docs.google.com/spreadsheets/d/1adlI3M5eCf7YoNnrfdJ7aEoxPWz8t08RQFkUyaSBPkQ/edit#gid=1046648178&range=182:196

I expected the no dns flush run to be faster in domainLookupTime, which it is, significantly.

However I didn't expect PrefetchFromHTTPS to improve the domainLookupTime which to my understanding is the dns time for the main page, and thus shouldn't have been affected.
https://www.w3.org/TR/navigation-timing/#dom-performancetiming-domainlookupstart

I suggest these next steps:
-Collect aggregate domainLookup time (i.e. for all resources) with and without the PrefetchFromHTTPS pref
https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/domainLookupStart
-Introduce network latency into the tests (currently the last-mile latency is very low)

I suggest these next steps:
-Collect aggregate domainLookup time (i.e. for all resources) with and without the PrefetchFromHTTPS pref
https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/domainLookupStart

Browsertime can be easily extended to capture this, for example in this change the total duration for each resource is collected:
https://gist.github.com/acreskeyMoz/c8249c839ffd5219b767e6fd5c89439b

Approximately one-third of all page loads include dns-prefetch so even a minimal performance improvement probably makes this important to fix (assuiming we are good on any privacy-related concerns).

Sorry for a delay here.

We found out that our dns measurements can be a bit wrong:

  • we perform a dns lookup
  • the response is posted to the socket thread thac can immediately use itt and make a connection. The socket thread post an event to the main thread
  • when the eevent is received on the main thread we measure time. So for each measurement we add extra time and if the main thread is busy that will be reflected in the measurements. At the same time the socket uses the dns results and does nott wait for this main thread hope.

We should fix the measurement and rerun experiments. This is bug 1626958.

Flags: needinfo?(dd.mozilla)

Is anyone working on this? DNS query times/spec DNS fetch/preconnect/resource hints are considered standard best practices in Chrome/Safari to reduce time to first paint by now.

I agree that this is important - I'll will ask if anyone from performance team is available.

Performance Impact: --- → P2
Keywords: perf:pageload
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.