Closed Bug 1728073 Opened 3 years ago Closed 3 years ago

With DoH enabled, Firefox doesn't read large hosts file

Categories

(Core :: Networking: DNS, defect, P3)

Product:
Core
Component:
Networking: DNS
Version:
Firefox 91
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
93 Branch
Tracking Flags:
Tracking Status
firefox93 --- fixed

People

(Reporter: nicoenkarin, Assigned: kershaw)

Details

(Whiteboard: [necko-triaged])

Attachments

(5 files)

1.log.txt.moz_log.zip
9.54 KB, application/octet-stream
Details
2.log.txt.moz_log.zip
325.91 KB, application/octet-stream
Details
3.log.txt.moz_log.zip
326.43 KB, application/octet-stream
Details
Bug 1728073 - Don't stop parsing if we encountered an error, r=#necko
48 bytes, text/x-phabricator-request
Details | Review
Screenshot 2021-09-02 at 15.37.42.png
12.08 KB, image/png
Details

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

  1. Enabled DoH

  2. Modified preferences:

    network.trr.blocklist_cleanup_done ---> true
    network.trr.custom_uri --- > https://dns.quad9.net/dns-query
    network.trr.mode --- > 2
    network.trr.request_timeout_ms --- > 4000
    network.trr.uri --- > https://dns.quad9.net/dns-query

  3. Added 0.0.0.0 www.google.com to the hosts file (~42000 entries).

Actual results:

  1. After restarting Firefox, I was still able to access www.google.com (even in a fresh profile).
  2. When I test with with a hosts file with only 0.0.0.0 www.google.com in it, it's blocked, however it takes some time to display the error message.
    about:networking#dnslookuptool then resolves to 0.0.0.0.
  3. With DoH disabled, or in another browser without DoH, www.google.com was blocked.
  4. Also in Chromium-dev 95.0.4626.0 with DoH enabled (same provider) www.google.com was blocked.

Expected results:

www.google.com should not load.

System: Linux Mint 19.3 64 bit
Firefox 91.0.2 from https://ftp.mozilla.org/pub/firefox/releases/

Should have been resolved with:
https://bugzilla.mozilla.org/show_bug.cgi?id=1616252

The Bugbug bot thinks this bug should belong to the 'Core::Networking: DNS' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → Networking: DNS
Product: Firefox → Core

I can't reproduce this locally. Here is what I did:

  1. Use perl -E 'for (1..50000) { say "127.0.0.10 $_.science" }' >> /etc/hosts to create a file with 50000 entries.
  2. Add 0.0.0.0 www.google.com in /etc/hosts.
  3. Visis www.google.com and it's blocked.

Could you try to capture a log by only setting MOZ_LOG to nsHostResolver:5?
Note that the log will expose the content in /etc/hosts, so if you want to keep it privately, please send it to my email directly.

Thanks.

Flags: needinfo?(nicoenkarin)
Attached file 1.log.txt.moz_log.zip —

Log without changes to hosts file. Added www.google.com

Flags: needinfo?(nicoenkarin)
Attached file 2.log.txt.moz_log.zip —

Removed 25 entries like:
$IP-address www.example.com

Attached file 3.log.txt.moz_log.zip —

Same as 2.log.txt.moz_log.zip, but launching Firefox in default profile.
(the first 2 logs were from Firefox launching in a clean profile).

Note, that it took much longer to display error message when accessing www.google.com.

It seems entries like $IP-address www.example.com cause the failing parsing of the hosts file here.

(In reply to nicoenkarin from comment #7)

It seems entries like $IP-address www.example.com cause the failing parsing of the hosts file here.

Could you share the hosts file that Firefox can't parse it successfully? What's the difference of entries like $IP-address www.example.com and others?

Thanks.

Flags: needinfo?(nicoenkarin)

(In reply to Kershaw Chang [:kershaw] from comment #8)

(In reply to nicoenkarin from comment #7)

It seems entries like $IP-address www.example.com cause the failing parsing of the hosts file here.

Could you share the hosts file that Firefox can't parse it successfully? What's the difference of entries like $IP-address www.example.com and others?

Thanks.

I've sent it via mail.

It's entries like:

172.67.207.184 www.universetoday.com
104.21.93.45 www.ghacks.net
185.172.149.132 mk0ghacksnety2pjrgh8.kinstacdn.com
31.7.187.157 forum.palemoon.org
140.211.166.86 forums.mozillazine.org
149.210.150.201 forum.linuxmintnl.nl

As I said earlier, Chromium has no problems with that.

Flags: needinfo?(nicoenkarin)
Assignee: nobody → kershaw
Pushed by kjang@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9538394f167e
Don't stop parsing if we encountered an error, r=necko-reviewers,dragana

Changing severity to S4 because of this is very uncommon issue and it is cause by unexpected characters in a hostfile. It can be fixed by users as well.

Severity: -- → S4
Priority: -- → P3
Whiteboard: [necko-triaged]

(In reply to Dragana Damjanovic [:dragana] from comment #12)

Changing severity to S4 because of this is very uncommon issue and it is cause by unexpected characters in a hostfile. It can be fixed by users as well.

What unexpected characters, if I may ask?
Then I can fix this here.

(In reply to nicoenkarin from comment #13)

(In reply to Dragana Damjanovic [:dragana] from comment #12)

Changing severity to S4 because of this is very uncommon issue and it is cause by unexpected characters in a hostfile. It can be fixed by users as well.

What unexpected characters, if I may ask?
Then I can fix this here.

It's in the fourth line of the hosts file you provided.
It should work if you delete this line.

(In reply to Kershaw Chang [:kershaw] from comment #14)

(In reply to nicoenkarin from comment #13)

(In reply to Dragana Damjanovic [:dragana] from comment #12)

Changing severity to S4 because of this is very uncommon issue and it is cause by unexpected characters in a hostfile. It can be fixed by users as well.

What unexpected characters, if I may ask?
Then I can fix this here.

It's in the fourth line of the hosts file you provided.
It should work if you delete this line.

I changed that line and that worked, but it took a long time to display the actual error page...
What could cause that?

And I thought lines that are commented out, were not even read.

Other browser without DoH displayed that page instantly and Chromium with DoH as well.

(In reply to nicoenkarin from comment #16)

(In reply to Kershaw Chang [:kershaw] from comment #14)

(In reply to nicoenkarin from comment #13)

(In reply to Dragana Damjanovic [:dragana] from comment #12)

Changing severity to S4 because of this is very uncommon issue and it is cause by unexpected characters in a hostfile. It can be fixed by users as well.

What unexpected characters, if I may ask?
Then I can fix this here.

It's in the fourth line of the hosts file you provided.
It should work if you delete this line.

I changed that line and that worked, but it took a long time to display the actual error page...
What could cause that?

Not sure. Could you capture a full http log (with MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5) again?
Thanks.

And I thought lines that are commented out, were not even read.

That's because our parser reads every line and then parse it.

Flags: needinfo?(nicoenkarin)

(In reply to Kershaw Chang [:kershaw] from comment #17)

(In reply to nicoenkarin from comment #16)

(In reply to Kershaw Chang [:kershaw] from comment #14)

(In reply to nicoenkarin from comment #13)

(In reply to Dragana Damjanovic [:dragana] from comment #12)

Changing severity to S4 because of this is very uncommon issue and it is cause by unexpected characters in a hostfile. It can be fixed by users as well.

What unexpected characters, if I may ask?
Then I can fix this here.

It's in the fourth line of the hosts file you provided.
It should work if you delete this line.

I changed that line and that worked, but it took a long time to display the actual error page...
What could cause that?

Not sure. Could you capture a full http log (with MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5) again?
Thanks.

And I thought lines that are commented out, were not even read.

That's because our parser reads every line and then parse it.

Thanks.

Hmm... This time it was way faster for some reason (I deleted the cache and closed Firefox first).
But here's the log file anyway (in your mail).

Flags: needinfo?(nicoenkarin)

https://hg.mozilla.org/mozilla-central/rev/9538394f167e

Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
status-firefox93: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 93 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: