Closed Bug 1596935 Opened 5 years ago Closed 3 months ago

Firefox doesn’t resolve <link rel=dns-prefetch> on HTTPS

Categories

(Core :: DOM: Networking, enhancement, P3)

70 Branch
enhancement

Tracking

()

RESOLVED FIXED
127 Branch
Performance Impact medium
Tracking Status
relnote-firefox --- 127+
firefox127 --- fixed

People

(Reporter: code, Assigned: acreskey)

References

Details

(Keywords: perf:pageload, Whiteboard: [necko-triaged] [necko-priority-queue])

Attachments

(3 files, 1 obsolete file)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36

Steps to reproduce:

  1. Add <link rel="dns-prefetch" href="https://mozilla.org/"> to a document loaded over HTTPS

Actual results:

Nothing. The preference network.dns.disablePrefetchFromHTTPS defaults to true and blocks this from working. It works as expected on HTTP, but HTTPS is kind of the default now.

Expected results:

The DNS for mozilla.org should be resolved early. <link rel="preconnect" href="https://mozilla.org/"> works on HTTPS and it “leaks” DNS resolution just the same as rel="dns-prefetch". The policy is inconsistent. The preference for network.dns.disablePrefetchFromHTTPS should be changed to false by default.

Chrome (Android, desktop) and Safari (macOS only) supports <link rel="dns-prefetch"> on HTTPS origins by default. (Safari on iOS doesn’t support it at all.)

This was a regression, introduced by bug 1572505.
smaug, was this overseen in the review or we really wanted to disable dns-prefetch?

Flags: needinfo?(bugs)

This is not a regression.

I’ve tested versions all the way back to 30 (my PC won’t run older versions) and dns-prefetch doesn’t work in any version on HTTPS. Works fine on HTTP in all versions.

In Chrome and Safari, explicit lookups with <link rel="dns-prefetch"> works for both HTTPS and HTTP documents. In Firefox, it only works on HTTP. This bug is a request to change Firefox behavior so explicit lookups also works on HTTPS.

I believed “network.dns.disablePrefetchFromHTTPS” controlled that behavior, as setting it to TRUE makes that change happen. I’m sorry for the confusion if that setting also controls other behavior.

I presume there was a reason Chrome didn't prefetch https back then (and we followed them); what changed or what did people find out that changed the analysis? Perhaps a privacy issue?

I presume there was a reason Chrome didn't prefetch https back then

I can’t find any evidence there has ever been a test for the protocol when resolving <link rel="dns-prefetch"> in either WebKit or Chromium sources. I’ve dug through different versions of their source code all the way back to 2008. It was unintentionally broken on protocols other than HTTP in WebKit for a few months in 2010 (this was after Firefox had implemented the feature).

If I was a betting man, I’d guess that this document is the source of the confusion. It was referenced in bug #453403 (tracking the feature implementation in Firefox). The document can be read in a way that suggests that Chrome doesn’t resolve <link rel="dns-prefetch"> hints on HTTPS websites. (It doesn’t say anything about it, though.) Comments in that bug reference this behaviour. The feature implementer wrote a blog post about his work at the time. It also references this document and behavior.
https://dev.chromium.org/developers/design-documents/dns-prefetching
https://bitsup.blogspot.com/2008/11/dns-prefetching-for-firefox.html

Firefox not resolving dns-prefetch on HTTPS origins seems to just have been a misunderstanding.

Summary: Switch network.dns.disablePrefetchFromHTTPS default to false → Firefox doesn’t resolve <link rel=dns-prefetch> on HTTPS
Priority: -- → P2

This came out of bug 492196 comment 16 I suspect. It seems there were tracking concerns at the time, but I'm not sure how those are different from <link rel=stylesheet> or some such for HTTPS.

Flags: needinfo?(bzbarsky)

No, that just refactored some code. The origin of this pref and its default value is bug 453403, which is the original landing of DNS prefetch.

Bug 453403 comment 17 mentions "Do we want something akin to chromium's https thing here" but I don't see in that bug what that thing was, nor do I recall what I was talking about there; sorry, it's been 11 years. :)

The tail end of bug 453403 comment 18 does sound like at the time Chromium did not do DNS prefetch for https.

I guess all that is summarized in comment 7.

In terms of what we should do now... Martin, do you have any opinions here?

Flags: needinfo?(bzbarsky) → needinfo?(mt)
Status: UNCONFIRMED → NEW
Ever confirmed: true

How about enabling dns-prefetch and preconnect only if DoH is used?

How about enabling dns-prefetch and preconnect only if DoH is used?

Well, you can resolve DNS (and open a TCP connection and perform a TLS handshake) using <link rel="preconnect">. This is supported in Firefox on HTTP and HTTPS websites. I don’t see why <link rel="dns-prefetch"> would be any more of a privacy problem on a HTTPS page than preconnect, prefetch, preload, stylesheet, or any other link relationship that lets website load external content. <img src> would also let you resolve an arbitrary domain name from a HTTPS page.

It seems there were tracking concerns at the time […]

The (notably ten years old and possibly out of date) Chrome design document talks about disabling DNS-prefetching of domains for <a href> on hover. I can’t find anything on the web with anyone talking about privacy issues with the <link rel="dns-prefetch"> mechanism.

I'd have to do some analysis to determine what the privacy consequences of prefetching would look like. Those are not completely obvious. My intuition is that Daniel's assertion in comment 11 is probably correct, but we should do due diligence.

Flags: needinfo?(mt)

Note: knowing now (I thought it was on by default all this time) that dns-prefetch has not been used by default, it would be worth going through the code and checking if we do all security checks properly, if principals are correct, etc. before turning it on by default. Also making sure we have tests. (Maybe we do, I do not know, but there are probably people that knows)

Component: DOM: Core & HTML → DOM: Networking
Priority: P2 → P3
Whiteboard: [necko-triaged]
See Also: → 1583298

I don't have any evidence that it's related to dns-prefetch, but in Bug 1583298 I did collect some results where spend more time in DNS lookup than Chrome.

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #14)

I don't have any evidence that it's related to dns-prefetch, but in Bug 1583298 I did collect some results where spend more time in DNS lookup than Chrome.

How mush work would it be to retest, maybe just a part, with pref turn on?
Probably it would be faster to look at web sites and see if they send rel=dns-prefetch

(In reply to Dragana Damjanovic [:dragana] from comment #15)

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #14)

I don't have any evidence that it's related to dns-prefetch, but in Bug 1583298 I did collect some results where spend more time in DNS lookup than Chrome.

How mush work would it be to retest, maybe just a part, with pref turn on?
Probably it would be faster to look at web sites and see if they send rel=dns-prefetch

Sorry, I missed this question.
I would be happy to re-test this. I have a bit of a testing queue at the moment, so it will have to wait a little bit.
I didn't realize this was switched by pref (network.dns.disablePrefetchFromHTTPS I assume?)

One thing I'm a bit concerned about in local testing is that we make use of the OS's DNS cache, right?
This could mask the impact of the change.
So perhaps I should clear the OS DNS cache between pageloads?

Flags: needinfo?(dd.mozilla)

I ran two identical tests comparing baseline with network.dns.disablePrefetchFromHTTPS set to false
Each of the tested sites make use of rel="dns-prefetch" via https.
These are live site tests and in between each cold page load I also flush the Windows DNS cache via ipconfig

From eyeballing it, there may be wins on some sites, although they are not necessarily consistent run-to-run.
For the 2nd run (on the 2nd tab), I removed some of the noisier sites:
https://docs.google.com/spreadsheets/d/1adlI3M5eCf7YoNnrfdJ7aEoxPWz8t08RQFkUyaSBPkQ/edit#gid=736102674

I added additional metrics, including domainLookupTime, (which is domainLookupEnd - domainLookupStart).
I also compared against a run where I did not flush the OS dns cache between cold page loads.

https://docs.google.com/spreadsheets/d/1adlI3M5eCf7YoNnrfdJ7aEoxPWz8t08RQFkUyaSBPkQ/edit#gid=1046648178&range=182:196

I expected the no dns flush run to be faster in domainLookupTime, which it is, significantly.

However I didn't expect PrefetchFromHTTPS to improve the domainLookupTime which to my understanding is the dns time for the main page, and thus shouldn't have been affected.
https://www.w3.org/TR/navigation-timing/#dom-performancetiming-domainlookupstart

I suggest these next steps:
-Collect aggregate domainLookup time (i.e. for all resources) with and without the PrefetchFromHTTPS pref
https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/domainLookupStart
-Introduce network latency into the tests (currently the last-mile latency is very low)

I suggest these next steps:
-Collect aggregate domainLookup time (i.e. for all resources) with and without the PrefetchFromHTTPS pref
https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/domainLookupStart

Browsertime can be easily extended to capture this, for example in this change the total duration for each resource is collected:
https://gist.github.com/acreskeyMoz/c8249c839ffd5219b767e6fd5c89439b

Approximately one-third of all page loads include dns-prefetch so even a minimal performance improvement probably makes this important to fix (assuiming we are good on any privacy-related concerns).

Sorry for a delay here.

We found out that our dns measurements can be a bit wrong:

  • we perform a dns lookup
  • the response is posted to the socket thread thac can immediately use itt and make a connection. The socket thread post an event to the main thread
  • when the eevent is received on the main thread we measure time. So for each measurement we add extra time and if the main thread is busy that will be reflected in the measurements. At the same time the socket uses the dns results and does nott wait for this main thread hope.

We should fix the measurement and rerun experiments. This is bug 1626958.

Flags: needinfo?(dd.mozilla)

Is anyone working on this? DNS query times/spec DNS fetch/preconnect/resource hints are considered standard best practices in Chrome/Safari to reduce time to first paint by now.

I agree that this is important - I'll will ask if anyone from performance team is available.

Performance Impact: --- → P2
Keywords: perf:pageload
Severity: normal → S3

This got stalled but it seems like an easy win if we are agreed on the privacy-side.

And it's not clear to me if there are actually any privacy concerns as we already support rel=preconnect and rel=preload, so will need someone with expertise to weight in here.

We are the only major browser to not support dns-prefetch on https.
https://caniuse.com/?search=dns-prefetch

Assignee: nobody → acreskey

According to Chrome Platform Status, dns-prefetch is quite common, appearing on about 20% of sites, 35% of top sites:
https://chromestatus.com/metrics/feature/timeline/popularity/899

I looked into the implementation of this and even if there are no privacy concerns, I don't think it's ready to ship as-is.
dns-prefetch is indeed commonly used, I see it on wikipedia.org, twitter.com, amazon.com, etc.
However when I enable it and profile (using https://en.wikipedia.org/wiki/Main_Page), I see two problems:

1 - The DNSPrefetch messages are stored in the content process and only sent when the document is loaded:

mozilla::dom::DeferredDNSPrefetches::SubmitQueueEntry()
mozilla::dom::DeferredDNSPrefetches::SubmitQueue()
mozilla::dom::DeferredDNSPrefetches::OnStateChange()
nsDocLoader::DoFireOnStateChange()
nsDocLoader::doStopDocumentLoad()
nsDocLoader::DocLoaderIsEmpty()

Intuitively, this doesn't seem ideal as I think we'd want to initiate the DNS requests asap to get the benefit of this attribute.

2 - For https://en.wikipedia.org/wiki/Main_Page we send about 300 DNS requests from child process to parent.
I added a temporary profiler marker and you can see them here, right after the document load completes.
https://share.firefox.dev/49Daj4V
So we'll have to find out what's happening there.

The deferal was introduced in https://hg.mozilla.org/mozilla-central/rev/a0c0ed9f461fc39635b2e99117fa3c1b85505ecd
The cause for it seems to have been related to shutdown leaks, but I can't figure out if we're still bound by the same limitations.

Thanks, the deferral code may make good sense for one of the DNS prefetch sources:

I see DNS prefetching coming from two places in the DOM:

  1. There is a DNS prefetch for every HTMLAnchorElement. I would call this something like "Speculative DNS"
    https://searchfox.org/mozilla-central/rev/da49863c3d6f34038d00f5ba701b9a2ad9cbadba/dom/html/HTMLAnchorElement.cpp#72,198

  2. And we have the link element, HTMLLinkElement, link rel="dns-prefetch"
    https://searchfox.org/mozilla-central/rev/da49863c3d6f34038d00f5ba701b9a2ad9cbadba/dom/html/HTMLLinkElement.cpp#551-553

Request from both routes are made as low priority DNS requests and only once the page has loaded.
https://searchfox.org/mozilla-central/rev/da49863c3d6f34038d00f5ba701b9a2ad9cbadba/dom/html/HTMLDNSPrefetch.cpp#368

For en.wikipedia.org, we get a little over 300 DNS prefetch from 1, anchors making speculative dns calls and just the two from 2, the rel="dns-prefetch" link elements in the dom.

<link rel="dns-prefetch" href="//meta.wikimedia.org" />
<link rel="dns-prefetch" href="//login.wikimedia.org">

And currently all of these are all skipped when on an https document.

The deferal code, seems like it would make sense for the HTMLAnchorElement's speculative DNS requests.
(Whether or not sending 300 speculative DNS requests to the parent is a good idea is another question).

But I propose this as next steps:

  • We separate the speculative dns behaviour of the HTMLAnchorElement from the dns-prefetch behaviour (defined here)
  • For dns-prefetch we don't defer the DNS requests until after pageload. i.e. we make them asap

We can then look at enable dns-prefetch on https pages.

WIP:
• Seperate html anchor "speculative dns" from rel=<dns-prefetch>
• Execute dns-prefetch requests right away

Split the rel=dns-prefetch behaviour from the Dom's speculative DNS prefetch for all anchor elements.
This will allow us to independently enable rel=dns-prefetch for https document without prefetching DNS for anchors.

This patch does not change the behaviour for either source of DNS prefetch.

Enable rel=dns-prefetch on HTTPS documents.

Depends on D205630

Attachment #9382163 - Attachment is obsolete: true

Per discussion, moving to priority queue.

Whiteboard: [necko-triaged] → [necko-triaged] [necko-priority-queue]
Attachment #9394871 - Attachment description: WIP: Bug 1596935 - Add tests for rel="dns-prefetch" → Bug 1596935 - Add tests for rel="dns-prefetch" r=#necko-reviewers
Attachment #9393152 - Attachment description: Bug 1596935 - Separate rel=dns-prefect behaviour from html anchor dns prefect r=#dom-core,#necko-reviewers → Bug 1596935 - Separate rel=dns-prefetch behaviour from html anchor dns prefetch r=#dom-core,#necko-reviewers
Pushed by acreskey@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c690fafd39c1
Separate rel=dns-prefetch behaviour from html anchor dns prefetch r=dom-core,necko-reviewers,sefeng
https://hg.mozilla.org/integration/autoland/rev/5de703ecb01b
Firefox doesn’t resolve <link rel=dns-prefetch> on HTTPS r=dom-core,necko-reviewers,kershaw,sefeng
https://hg.mozilla.org/integration/autoland/rev/57f6925a520c
Add tests for rel="dns-prefetch" r=necko-reviewers,valentin
Regressions: 1892047
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → 127 Branch

Release Note Request (optional, but appreciated)
[Why is this notable]:
In Firefox 127, we completed work to optimize and enable DNS prefetching for HTTPS documents via the rel="dns-prefetch" link hint. This standard allows web developers to specify domain names for important assets that should be resolved preemptively.

Firefox was the only major browsers not supporting rel="dns-prefetch" on HTTPS pages, despite the standard permitting it. In Fx127, we closed this gap by implementing DNS prefetching for secure contexts.
[Affects Firefox for Android]: Yes
[Suggested wording]:
In Firefox 127, we completed work to optimize and enable DNS prefetching for HTTPS documents via the rel="dns-prefetch" link hint. This standard allows web developers to specify domain names for important assets that should be resolved preemptively.
[Links (documentation, blog post, etc)]:

relnote-firefox: --- → ?

Note added to our Nightly 127 release notes.
https://www.mozilla.org/en-US/firefox/127.0a1/releasenotes/

Type: defect → enhancement

We currently have an enterprise policy for NetworkPrediction:

https://mozilla.github.io/policy-templates/#networkprediction

It corresponds to this policy in Chrome:

https://chromeenterprise.google/policies/#NetworkPredictionOptions

It sets these prefs:

  setAndLockPref("network.dns.disablePrefetch", !param);
  setAndLockPref("network.dns.disablePrefetchFromHTTPS", !param);

What additional things should be added to this policy as a result of this bug?

(see https://github.com/mozilla/policy-templates/issues/1125)

It's a bit nuanced.
The behaviour that I would categorize as "Predictive" is the dom action of prefetching DNS records for every anchor element in the document.

As a result of this bug, that is now controlled by these two prefs:

dom.prefetch_dns_for_anchor_http_document (default: true)
dom.prefetch_dns_for_anchor_https_document (default: false)

https://searchfox.org/mozilla-central/rev/a67f92611070fcdba22a106ee140ab0059cc4611/modules/libpref/init/StaticPrefList.yaml#3345,3351
See: https://phabricator.services.mozilla.com/D205630

I think both prefs should be disabled if the NetworkPrediction enterprise policy is enabled.

And that leaves the existing prefs, which now control rel=dns-prefetch behaviour.

network.dns.disablePrefetch
network.dns.disablePrefetchFromHTTPS

I wouldn't personally consider these to be "predictive", since they are following the spec by preemptively (but not predictively) performing the dns resolution.
https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/dns-prefetch

But maybe for enterprise policies it's better to aim for a conservative approach (and fewer changes), and so these prefs should still be set?

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: