1766052 - Fails to launch when system-side DNS over TLS is enabled, no console output

Status: RESOLVED INCOMPLETE

(Firefox :: General, defect)

Description

[mjevange]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:100.0) Gecko/20100101 Firefox/100.0

Steps to reproduce:

NOTE: Issue affects Stable as well

This was quite difficult to troubleshoot due to the fact that there is zero terminal output

1. Configure DNS over TLS

[/etc/systemd/resolved.conf]
DNS=1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com
FallbackDNS=9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
DNSSEC=yes
DNSOverTLS=no

2. Restart service
   ~$ sudo systemctl restart systemd-resolved

3. Test DNS over TLS (note "encrypted transport")
   ~$ resolvectl query sigok.verteiltesysteme.net

sigok.verteiltesysteme.net: 134.91.78.139 -- link: wlan0
2001:638:501:8efc::139 -- link: wlan0

-- Information acquired via protocol DNS in 685.4ms.
-- Data is authenticated: yes; Data was acquired via local or encrypted transport: yes
-- Data from: network

4. Launch Firefox
   ~$ firefox-developer-edition

Does not start, no logs in journalctl, no output in terminal

5. Disable DNS over TLS

[/etc/systemd/resolved.conf]
DNSSEC=yes
DNSOverTLS=no

6. Restart service
   ~$ sudo systemctl restart systemd-resolved

7. Test DNS over TLS (note "encrypted transport")
   ~$ resolvectl query sigok.verteiltesysteme.net

sigok.verteiltesysteme.net: 134.91.78.139 -- link: wlan0
2001:638:501:8efc::139 -- link: wlan0

-- Information acquired via protocol DNS in 216.0ms.
-- Data is authenticated: yes; Data was acquired via local or encrypted transport: no
-- Data from: network

8. Launch Firefox
   ~$ firefox-developer-edition

Starts fine

Actual results:

In Step 4, Firefox did not start or produce logging

Expected results:

In Step 4, Firefox should have started or produced logging

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Networking: DNS' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[mjevange]

Also tested on: 101.0a1

Comment 3

[Kershaw Chang [:kershaw]]

Hi, could you try to record some http log?
I think you could try to do this:

MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 ./firefox-developer-edition &>tmp.log

The log file tmp.log should contain some logs for debugging. Thanks.

Comment 4

[mjevange]

This doesn't seem to produce and logging. Neither in terminal, nor in tmp.log

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:kershaw, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 6

[Kershaw Chang [:kershaw]]

This looks like not a networking issue, since Firefox seems to be stuck at startup.
Change the component and see if someone else has an idea.

Comment 7

[Robin Steuber (she/her) [:bytesized]]

I gave this a try on Ubuntu 20.04, but I can't reproduce the problem.

I can't actually seem to get the DNS over TLS bit to work in the first place, using the resolved.conf that you provided.

$ resolvectl query sigok.verteiltesysteme.net
sigok.verteiltesysteme.net: resolve call failed: DNSSEC validation failed: failed-auxiliary

I checked that firefox does launch anyways.

Is there any more information you can give so we can try to figure out what's going on here?

Comment 8

[Robin Steuber (she/her) [:bytesized]]

Nothing more we can do about this without a response from the user.

Comment 9

[mjevange]

(In reply to Kirk Steuber (he/him) [:bytesized] from comment #7)

I gave this a try on Ubuntu 20.04, but I can't reproduce the problem.

I can't actually seem to get the DNS over TLS bit to work in the first place, using the resolved.conf that you provided.

$ resolvectl query sigok.verteiltesysteme.net
sigok.verteiltesysteme.net: resolve call failed: DNSSEC validation failed: failed-auxiliary

I checked that firefox does launch anyways.

Is there any more information you can give so we can try to figure out what's going on here?

(In reply to Kirk Steuber (he/him) [:bytesized] from comment #7)

I gave this a try on Ubuntu 20.04, but I can't reproduce the problem.

I can't actually seem to get the DNS over TLS bit to work in the first place, using the resolved.conf that you provided.

$ resolvectl query sigok.verteiltesysteme.net
sigok.verteiltesysteme.net: resolve call failed: DNSSEC validation failed: failed-auxiliary

I checked that firefox does launch anyways.

Is there any more information you can give so we can try to figure out what's going on here?

Can we please re-open,

To reproduce the bug you must first properly configure DNS over TLS.
I've not done this with Ubuntu (I did Arch), but it should be the exact same.

Try this:
https://support.quad9.net/hc/en-us/articles/4409119438349-DNS-over-TLS-Ubuntu-20-04-Linux-Mint-20-3-Native-

Comment 10

[Robin Steuber (she/her) [:bytesized]]

I'll see if I can reproduce with that.

Comment 11

[Robin Steuber (she/her) [:bytesized]]

(In reply to mjevange from comment #0)

1. Configure DNS over TLS

[/etc/systemd/resolved.conf]
DNS=1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com
FallbackDNS=9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
DNSSEC=yes
DNSOverTLS=no

Is DNSOverTLS meant to be yes in step 1?

---

I don't seem to be getting transport encryption information from resolvectl, for whatever reason. My output just looks like this:

> resolvectl query sigok.verteiltesysteme.net
sigok.verteiltesysteme.net: 134.91.78.139      -- link: ens33

-- Information acquired via protocol DNS in 394.5ms.
-- Data is authenticated: yes

I do see that I'm getting DNS information over port 853, which presumably means that DNS over TLS is working?

But I still am not seeing any problem with Firefox. I tried the current Ubuntu packaged build and a local build, and both seem to launch with no problems.

Can you give any more information that might help me reproduce the problem? If you have access to a VM, it would be nice if you could tell me how to reproduce the problem on a clean VM so that you can tell me exactly what steps are necessary to reproduce.

Comment 12

[mjevange]

(In reply to Kirk Steuber (he/him) [:bytesized] from comment #11)

Is DNSOverTLS meant to be yes in step 1?

OMG yes, sorry!

DNSOverTLS needs to be set to yes in order to reproduce the issue!

To confirm that DNSOverTLS is functioning, we should see Data was acquired via local or encrypted transport: yes
(Unless Ubuntu's version of resolved does not include this function maybe)

Comment 13

[Robin Steuber (she/her) [:bytesized]]

Yeah, I set DNSOverTLS to yes. But I notice that even when you have it turned off, your copy of resolvectl says

Data was acquired via local or encrypted transport: no

And I'm not seeing that line at all. But unencrypted DNS isn't typically sent over port 853, so I assume that it's working.

---

I strongly suspect that something other than Firefox is causing this problem. Firefox should typically log all sorts of stuff to the console, especially if something fatal happens. The fact that nothing is being logged suggests to me that Firefox is being prevented from running at all. I don't know enough about your system to know why this might be. But if Firefox is being prevented from running, I doubt that any change that we make to Firefox can fix the problem. Whatever is causing that needs to be addressed.

If there is anything else that you want me to try to see if I can reproduce your problem so I can try to see what's going on for myself, I'd be happy to try it. But, otherwise, I'm not sure what else I can do to help here.

Comment 14

[Robin Steuber (she/her) [:bytesized]]

I'm going to close this because we were never able to get enough information for it to be actionable. If anyone is able to provide more information, we can certainly reopen it and take another look.