1856421 - Firefox cannot resolve host on refresh

Status: NEW

(Core :: Networking, defect, P2)

Description

[Valerio Aimale]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0

Steps to reproduce:

Have a HTTPS site with Basic authentication, and refresh set
<meta http-equiv="refresh" content="300" />
in head.

Actual results:

First request is successfully transferred.
https://myhost/munin/home/another_host/index.html#nginx
Host: myhost
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Authorization: Basic dmFsZXJwerwe3bGFwaXNvbmE=
Connection: keep-alive
Referer: https://myhost/munin/
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
GET: HTTP/2.0 200
server: nginx
date: Mon, 02 Oct 2023 17:27:00 GMT
content-type: text/html
content-length: 28537
last-modified: Mon, 02 Oct 2023 17:25:26 GMT
etag: "651afd06-6f79"
accept-ranges: bytes
X-Firefox-Spdy: h2

The folowing refresh request fails with NS_ERROR_UNKNOWN_HOST - I have no resolver failure. And my Firefox runs on a Mac, which has cached resolver entries via mDNS.

https://myhost/munin/home/another_host/index.html#nginx
Host: myhost
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/118.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Authorization: Basic dmFsZXJwerwe3bGFwaXNvbmE=
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
NS_ERROR_UNKNOWN_HOST

Expected results:

Page should refresh without host resolution failure. I have tested the very same page with Chrome and Safari: the page refreshes correctly.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Networking' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Sunil Mayya]

Thank you for filing the report.

Can you please provide http logs to help us debug this further.
You can email the logs to [e-mail] if you are concerned about saving sensitive personal information in the tool.

Comment 3

[Valerio Aimale]

I have provided the requested logfile via email.

Extra bit of info: firefox on Linux, in the same setting doe not display the same behavior. Firefox on Linux correctly retrieves the IP address.

Comment 4

[Valentin Gosu [:valentin] (he/him)]

Thank you for the logs.
It seems this is an issue that is caused by the DNS over HTTPS split horizon mitigations.
I can see that the domain resolves to both a public and a local IP address at times.
You can probably bypass this issue by going to about:preferences#privacy and set the DNS over HTTPS mode to Off.

I'll try to work out a fix soon.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

I have looked the logs, and it seems we do several requests for the DNS record with different flags:

2023-10-02 19:07:59.210473 UTC - [Parent 4963: Socket Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 0
2023-10-02 19:08:05.168116 UTC - [Parent 4963: Main Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 2
2023-10-02 19:08:05.235662 UTC - [Parent 4963: Socket Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 12000
2023-10-02 19:08:05.297603 UTC - [Parent 4963: Socket Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 12601
2023-10-02 19:13:14.827188 UTC - [Parent 4963: Socket Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 1
2023-10-02 19:13:14.867950 UTC - [Parent 4963: Socket Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 12021
2023-10-02 19:13:14.911983 UTC - [Parent 4963: Socket Thread]: D/nsHostResolver NameLookup: www.aimale.com effectiveTRRmode: 2 flags: 12081

I think this is caused when we try to load it over IPv6 but it fails with NS_ERROR_DEFINITIVE_UNKNOWN_HOST because the host did have an IPv4 record. The connection manager then sees this and does not fallback to regular DNS.

Comment 6

[Valerio Aimale]

My host has both an internal and external IPv4 resolved IP. It has an external IPv6 resolved IP, no internal IPv6

Comment 7

[Valentin Gosu [:valentin] (he/him)]

Regarding the flags for each request:

0: RESOLVE_DEFAULT_FLAGS
2: RESOLVE_CANONICAL_NAME
12000: RESOLVE_DISABLE_IPV6 | RESOLVE_OFFLINE | RESOLVE_DISABLE_IPV4 | RESOLVE_DISABLE_TRR | RESOLVE_REFRESH_CACHE | RESOLVE_TRR_DISABLED_MODE | RESOLVE_IGNORE_SOCKS_DNS
12601: RESOLVE_PRIORITY_LOW | RESOLVE_SPECULATE | RESOLVE_DISABLE_IPV6 | RESOLVE_ALLOW_NAME_COLLISION | RESOLVE_IGNORE_SOCKS_DNS
1: RESOLVE_BYPASS_CACHE
12021: RESOLVE_PRIORITY_MEDIUM | RESOLVE_SPECULATE | RESOLVE_DISABLE_IPV6 | RESOLVE_OFFLINE | RESOLVE_DISABLE_IPV4 | RESOLVE_DISABLE_TRR | RESOLVE_REFRESH_CACHE | RESOLVE_TRR_DISABLED_MODE | RESOLVE_IGNORE_SOCKS_DNS
12081: RESOLVE_SPECULATE | RESOLVE_DISABLE_IPV6 | RESOLVE_ALLOW_NAME_COLLISION | RESOLVE_DISABLE_TRR | RESOLVE_REFRESH_CACHE | RESOLVE_TRR_DISABLED_MODE | RESOLVE_IGNORE_SOCKS_DNS

One thing that stands out is that some requests have both RESOLVE_DISABLE_IPV6 and RESOLVE_DISABLE_IPV4 flags. That seems like a bug. Not sure if it's the direct cause of this issue or not.

(In reply to Valerio Aimale from comment #6)

My host has both an internal and external IPv4 resolved IP. It has an external IPv6 resolved IP, no internal IPv6

Thank you for that information. It will make it to build a reproducible test case for this.