1730418 - Blocked network requests still reach a custom DoH resolver

Status: RESOLVED FIXED

(Core :: Networking: DNS, defect, P2)

Description

[Phil]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:93.0) Gecko/20100101 Firefox/93.0

Steps to reproduce:

Firefox Developer Edition v93.0b3
uBlock Origin v1.37.2
DNS Resolver (browser): NextDNS via DoH (custom URL to identify FF)
DNS Resolver (macOS): NextDNS via NextDNS macOS app

In my uBlock settings, I have created dynamic filtering rules like:

• http://play.google.com/log * block
• https://play.google.com/log * block

Some additional things I've confirmed:

• The "Disable pre-fetching" setting in uBlock is enabled
• network.dns.disablePrefetch is set to true in about:config
• network.dns.disablePrefetchFromHTTPS is set to true in about:config (default)
• Firefox is configured to use the system proxy
• There are two enabled filter lists which add relevant rules
• EasyPrivacy (currently line 24594): @@||play.google.com/log?$xmlhttprequest,domain=google.com
• For me this rule is being overridden by the below rule
• AdGuard Tracking Protection (currently line 25002): ||play.google.com/log?$xmlhttprequest,redirect-rule=noop.txt
• For me this rule appears as enforced in the uBlock logger, alongside my custom rule enforcement
• Requests to play.google.com do not appear in about:networking#dns

This issue has been separately investigated by the uBlock team:
https://github.com/uBlockOrigin/uBlock-issues/issues/1710

Actual results:

I confirmed via both the uBlock logger and Firefox dev tools that all requests being made to the play.google.com domain match my blocking rule(s), and are showing as blocked. Still, I have also confirmed that when I visit mail.google.com, I am seeing DNS queries for play.google.com from the Firefox Developer Edition DoH URI in my NextDNS logs every few seconds.

Expected results:

I believe that a bug in Firefox itself causes pre-fetching to be allowed even with all relevant settings configured as desired.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Networking: DNS' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Phil]

Some of the markdown formatting in my original description may have made it unclear. Here is an updated version with corrected formatting.

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:93.0) Gecko/20100101 Firefox/93.0

Steps to reproduce:

Firefox Developer Edition v93.0b3
uBlock Origin v1.37.2
DNS Resolver (browser): NextDNS via DoH (custom URL to identify FF)
DNS Resolver (macOS): NextDNS via NextDNS macOS app

In my uBlock settings, I have created dynamic filtering rules like:

* http://play.google.com/log * block
* https://play.google.com/log * block

Some additional things I've confirmed:

• The "Disable pre-fetching" setting in uBlock is enabled
• network.dns.disablePrefetch is set to true in about:config
• network.dns.disablePrefetchFromHTTPS is set to true in about:config (default)
• Firefox is configured to use the system proxy
• There are two enabled filter lists which add relevant rules
  • EasyPrivacy (currently line 24594): @@||play.google.com/log?$xmlhttprequest,domain=google.com
    • For me this rule is being overridden by the below rule
  • AdGuard Tracking Protection (currently line 25002): ||play.google.com/log?$xmlhttprequest,redirect-rule=noop.txt
    • For me this rule appears as enforced in the uBlock logger, alongside my custom rule enforcement
• Queries for play.google.com do not appear in about:networking#dns

This issue has been separately investigated by the uBlock team: uBlock-issues#1710

Actual results:

I confirmed via both the uBlock logger and Firefox dev tools that all requests being made to the play.google.com domain match my blocking rule(s), and are showing as blocked. Still, I have also confirmed that when I visit mail.google.com, I am seeing DNS queries for play.google.com from the Firefox Developer Edition DoH URI in my NextDNS logs every few seconds.

Expected results:

I believe that a bug in Firefox itself causes pre-fetching to be allowed even with all relevant settings configured as desired.

Comment 3

[Valentin Gosu [:valentin] (he/him)]

Could you also disable the predictor and see if it helps? network.predictor.enabled to false

Comment 4

[gwarser]

uBlock Origin is using privacy api, so network.predictor.enabled is also disabled by "Disable pre-fetching" option.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

So, the question here is what is triggering the DNS resolutions for play.google.com
Does your nextDNS resolver actually respond to requests for play.google.com? Otherwise they would not show up in about:networking#dns

My best idea here is to set a breakpoint when resolving play.google.com and to check the backtrace 🙂
I need to make the above setup first.

Comment 6

[Phil]

(In reply to Valentin Gosu [:valentin] (he/him) from comment #3)

Could you also disable the predictor and see if it helps? network.predictor.enabled to false

I have the "Disable pre-fecthing" option selected in my uBlock origin settings, and I have confirmed that network.predictor.enabled is set to false in about:config, which aligns with the response from gwarser.

(In reply to Valentin Gosu [:valentin] (he/him) from comment #5)

Does your nextDNS resolver actually respond to requests for play.google.com? Otherwise they would not show up in about:networking#dns

Yes. As a DNS resolver, NextDNS provides domain-level blocking only. I do not want to block all DNS queries for play.google.com, since some may not be tracking-related. I only want to block requests made to play.google.com/log. When all requests to the play.google.com domain are blocked on a page (because they're all made to the /log path), I am still seeing DNS queries for the domain, and so NextDNS responds as required.

Comment 7

[yokoffing]

To add to this example: I have play.google.com hard-blocked in uBlock Origin (to allow no exceptions whatsoever). Firefox speculative connections are disabled.

When I navigate to gmail.com with network.trr.mode set to 2 or 3 and my custom NextDNS setup active, my NextDNS logs show a DNS query for play.google.com.

However, if I set network.trr.mode to 0 or 5, there are no queries to play.google.com in my logs.

We have ruled out any issues with uBlock Origin. We suspect Firefox is leaking DNS requests when DOH is active (network.trr.mode = 2 or 3).

Comment 8

[Valentin Gosu [:valentin] (he/him)]

OK, so I tried to reproduce the issue with mode and 3, and doh-proxy.
What I'm seeing is that the channels get blocked, but when using TRR we also resolve HTTPS records. We don't resolve HTTPS records using the native resolver, which is why this only occurs with TRR.

The HTTPS resolve is triggered from BeginConnect -> MaybeStartDNSPrefetch (which is probably before we get the chance to block the channel.

Kershaw, do you want to take this?

Comment 9

[Kershaw Chang [:kershaw]]

As a temporary workaround, I'd suggest to turn off network.dns.upgrade_with_https_rr.
Could you try to set network.dns.upgrade_with_https_rr to false and see if you still have this issue?

Comment 10

[Phil]

(In reply to Kershaw Chang [:kershaw] from comment #9)

As a temporary workaround, I'd suggest to turn off network.dns.upgrade_with_https_rr.
Could you try to set network.dns.upgrade_with_https_rr to false and see if you still have this issue?

After some brief testing, this workaround appears to resolve the issue!

Separately, I did a quick web search for the network.dns.upgrade_with_https_rr option, to try and get some more detail on what it does, but I wasn't able to find anything. Do you mind providing a brief explanation, so I can determine if I want to leave the setting set to false?

Comment 11

[Kershaw Chang [:kershaw]]

(In reply to Phil from comment #10)

(In reply to Kershaw Chang [:kershaw] from comment #9)

As a temporary workaround, I'd suggest to turn off network.dns.upgrade_with_https_rr.
Could you try to set network.dns.upgrade_with_https_rr to false and see if you still have this issue?

After some brief testing, this workaround appears to resolve the issue!

Separately, I did a quick web search for the network.dns.upgrade_with_https_rr option, to try and get some more detail on what it does, but I wasn't able to find anything. Do you mind providing a brief explanation, so I can determine if I want to leave the setting set to false?

You could find the information about HTTPS RR in this spec.
This pref is about the feature that does HTTPS upgrade if HTTPS RR is available.
Here is what the spec says:

   By publishing a usable HTTPS RR, the server operator indicates that
   all useful HTTP resources on that origin are reachable over HTTPS,
   similar to HTTP Strict Transport Security [HSTS].

Comment 12

[Phil]

(In reply to Kershaw Chang [:kershaw] from comment #11)

You could find the information about HTTPS RR in this spec.

Ah, some light reading. Thank you :)

This pref is about the feature that does HTTPS upgrade if HTTPS RR is available.

So my question then would be: Why does FF allow DNS queries for "useful" HTTP resources that are not available via HTTPS when using a custom DoH resolver? Perhaps that's the root of this issue?

Comment 13

[yokoffing]

(In reply to Kershaw Chang [:kershaw] from comment #9)

As a temporary workaround, I'd suggest to turn off network.dns.upgrade_with_https_rr.

Does setting this pref to false override network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc, or do these settings also need to be set to false until there's a bug fix?

Comment 14

[Kershaw Chang [:kershaw]]

Attachment: Bug 1730418 - Fetch HTTPS RR in nsHttpChannel::MaybeUseHTTPSRRForUpgrade, r=#necko

Comment 15

[Kershaw Chang [:kershaw]]

(In reply to yokoffing from comment #13)

(In reply to Kershaw Chang [:kershaw] from comment #9)

As a temporary workaround, I'd suggest to turn off network.dns.upgrade_with_https_rr.

Does setting this pref to false override network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc, or do these settings also need to be set to false until there's a bug fix?

No, this pref doesn't doesn't affect network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc. Only setting network.dns.upgrade_with_https_rr to false is enough.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/79a3346995bc
Fetch HTTPS RR in nsHttpChannel::MaybeUseHTTPSRRForUpgrade, r=necko-reviewers,dragana

Comment 17

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/79a3346995bc

Comment 18

[Phil]

For my own clarification, the attached changeset should correct the behavior cited in the original description without having to keep network.dns.upgrade_with_https_rr set to false, correct? And the changes are expected to be included in FF v94?

Comment 19

[Kershaw Chang [:kershaw]]

(In reply to Phil from comment #18)

For my own clarification, the attached changeset should correct the behavior cited in the original description without having to keep network.dns.upgrade_with_https_rr set to false, correct? And the changes are expected to be included in FF v94?

Yes, that's correct and the change will be include in 94.

Comment 20

[Phil]

(In reply to Kershaw Chang [:kershaw] from comment #19)

Yes, that's correct and the change will be include in 94.

Great! Thank you for the prompt attention and resolution!

Comment 21

[Alliz]

94.0.1a Nighty DNS Leak is actual problem using uBlock

Comment 22

[Kershaw Chang [:kershaw]]

(In reply to Alliz from comment #21)

94.0.1a Nighty DNS Leak is actual problem using uBlock

Sorry, I don't quite understand this.
Could you perhaps file another bug and provide a STR?

Thanks.