1543201 - DoH Privacy Enhancement: Do not set the User-Agent header for DoH requests

Status: RESOLVED FIXED

(Core :: Networking: DNS, enhancement, P3)

Description

[nusenu]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

Steps to reproduce:

It is great to see that Mozilla embraced DNS privacy.

We did pretty well without sending something like
a user-agent to the DNS resolver during the last decades
of plain old DNS operations.

To avoid unnecessary disclosure of user-agent data to
the DoH server, Firefox should not set the user-agent header for DoH requests.

The user-agent header could be used to further fingerprint and track the client.

The user-agent is not required as per RFC8484 and DNS resolvers
did fine without it until now.

https://tools.ietf.org/html/rfc8484#section-8.2

Comment 1

[Valentin Gosu [:valentin] (he/him)]

The DoH server already sees your DNS traffic and IP address, so I wouldn't be too worried about the UA string being a fingerprinting target :)
But in any case, we should probably not send it since it just adds extra useless bytes to every payload.

Comment 2

[Valentin Gosu [:valentin] (he/him)]

(In reply to Valentin Gosu [:valentin] from comment #1)

But in any case, we should probably not send it since it just adds extra useless bytes to every payload.

Actually, because of HPACK that probably doesn't apply. But we should strip the UA anyway.

Comment 3

[Chris Peterson [:cpeterson]]

Can the DoH server response set cookies on the client?

Comment 4

[Valentin Gosu [:valentin] (he/him)]

(In reply to Chris Peterson [:cpeterson] from comment #3)

Can the DoH server response set cookies on the client?

No. We explicitly check that we don't send any cookies for the DoH requests:

https://searchfox.org/mozilla-central/rev/69ace9da347adcc4a33c6fa3d8e074759b91068c/testing/xpcshell/moz-http2/moz-http2.js#640,652,753-756

Comment 5

[nusenu]

reference to similar enhancement requests (accept-language header): bug 1544724

Comment 6

[ahu]

I care about my privacy, and I would not like a DNS provider to know more about me than necessary. It is all the more fodder for tracking. So if you could find a way to disable this it would be nice. If some hints are provided I could give it a try to disable it myself. Please let me know!

Comment 7

[ahu]

I wrote a trivial patch that sets the User-Agent to 'DoH' for TRR requests. The reasoning behind setting it to DoH is that TRR is all about privacy. Previously, a DNS server had no idea what application its queries were coming from (let alone the exact version and operating system), it seems best to retain that privacy level. The patch is so trivial that I hope you can take it from here, as I am not familiar with the pull request/review/testing system of Mozilla:

--- a/netwerk/dns/TRR.cpp
+++ b/netwerk/dns/TRR.cpp
@@ -258,6 +258,10 @@ nsresult TRR::SendHTTPRequest() {
       NS_LITERAL_CSTRING("application/dns-message"), false);
   NS_ENSURE_SUCCESS(rv, rv);
 
+  rv = httpChannel->SetRequestHeader(NS_LITERAL_CSTRING("User-Agent"), NS_LITERAL_CSTRING("DoH"),
+                        false);
+  NS_ENSURE_SUCCESS(rv, rv);
+
   nsAutoCString cred;
   gTRRService->GetCredentials(cred);
   if (!cred.IsEmpty()) {

Please let me know if this works for you, or if I should familiarize myself with more of the merging ritual.

Thanks!

Comment 8

[nusenu]

Instead of setting it to an arbitrary string it would be great if the UA header was not set at all.
Since agreeing on a string across DoH implementations might be harder than not setting it at all and it saves
the "extra useless bytes to every payload".

Comment 9

[ahu]

I ran for a bit without a user-agent header, but it appears to confuse some infrastructure. Some sites did not load. Not sure if it is a mandatory header.

Comment 10

[nusenu]

"Some sites did not load." suggests that you refer to non-DoH HTTP requests.

I agree that you will run into issues when browsing the web without setting a user-agent header but that should not be the case with DoH servers.

Should there be any DoH server not replying to DoH clients not setting the UA header we can tell them to fix their stuff - since it is still relatively new.

If you have specific examples of DoH servers failing without UA header, all the better.

Comment 11

[ahu]

To really remove the header would require the infrastructure that bug 1544724 also needs. I'm no HTTP-ologist, so just let me know. Once we get 1544724 done, we have options.

Comment 12

[ahu]

(In reply to ahu from comment #9)

I ran for a bit without a user-agent header, but it appears to confuse some infrastructure. Some sites did not load. Not sure if it is a mandatory header.

To clarify, DoH servers work fine without a User-Agent header. It is random webservers hosting websites that get confused.

Comment 13

[Boris Zbarsky [:bzbarsky]]

Valentin, are you willing to guide comment 7 through to landing?

Comment 14

[Valentin Gosu [:valentin] (he/him)]

(In reply to ahu from comment #7)

Please let me know if this works for you, or if I should familiarize myself with more of the merging ritual.

Thank you for the patch.
Normally we accept contributions in the form of phabricator review requests. Here's a short guide:
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/How_to_Submit_a_Patch

Additionally I think a few changes are needed:

• We should clear the user agent header, not set it to DoH which seems fairly arbitrary.
• We should add a preference for this change. Some users might want this during testing, some DoH servers may (wrongly) fail without it.
• We should add a test for it (see test_trr.js )

Let me know if you need help with any of this.

Comment 15

[Valentin Gosu [:valentin] (he/him)]

Hi, I just pushed bug 1544724. The patch here should be very similar.
Let me know if you're up for doing it. Cheers!

Comment 16

[ahu]

It would be an honour! Thanks for doing the hard work, I'll get on it tomorrow (Thursday).

Comment 17

[Valentin Gosu [:valentin] (he/him)]

Great to have you on board :) I assigned the bug to you.
Let me know if I can help with anything.

Comment 18

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1543201 - DoH Privacy Enhancement: Do not set the User-Agent header for DoH requests r=dragana

Comment 19

[Valentin Gosu [:valentin] (he/him)]

Sorry for stealing the bug, but it would be nice to have it in this release.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/77171b4cd969
DoH Privacy Enhancement: Do not set the User-Agent header for DoH requests r=dragana

Comment 21

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/77171b4cd969