2000912 - Migrate to Necko nsIDNSService for TXT, MX, and SRV record lookups

Status: NEW

(MailNews Core :: Networking, defect)

Description

[Eleanor Dicharry]

We currently have a javascript wrapper to the native system libraries for DNS resolution and are seeing frequent crashes on MacOS in the native DNS resolution system libraries. It'd be better to use m-c's interface for DNS resolution since it's probably more tested and robust on multiple systems. However, Necko doesn't currently support SRV lookups, so we need to implement SRV lookups in Necko before we can use their DNS resolution classes. We checked with the Necko team and they said they'd be open to a contribution, so in order to switch over we need to:

1. Implement SRV and MX lookups in Necko. This has been discussed at length in multiple issues. See bug 14328 .
2. Once Necko supports SRV and MX lookups, replace the native C library wrapping in DNS.worker.mjs with the use of the Necko XPCOM interface for DNS resolution.

Comment 1

[Eleanor Dicharry]

CC :BenB and :clokep in case they have any questions, suggestions, or issues with the proposed implementation approach since they appear to have a good bit of history and experience in this area.

Comment 2

[Eleanor Dicharry]

We also need to support MX records.

Comment 3

[Patrick Cloke [:clokep]]

I don't have any other thoughts besides that we use that same code for MX and TXT messages. Not sure if this is just about SRV or not due to the title and description mismatching.

I think I looked at this in the past a bit, I'm sure you've found all this, but there's a few different implementations for DNS resolution (all in https://searchfox.org/firefox-main/source/netwerk/dns): native resolution, DNS-over-HTTP, and maybe another one? I think it also goes across IPC boundaries so there's a bunch of serialization/deserialization logic IIRC.

I would definitely focus on adding the few extra types that Thunderbird needs and not try to create some whole abstract support any type of DNS query.

Comment 4

[Patrick Cloke [:clokep]]

(Also very exciting to have someone working on this.)

Comment 5

[Eleanor Dicharry]

(In reply to Patrick Cloke [:clokep] from comment #3)

I don't have any other thoughts besides that we use that same code for MX and TXT messages. Not sure if this is just about SRV or not due to the title and description mismatching.

Yes, but we still need the ability to send the MX query in Necko even if it's the same parsing as TXT for the DNS RR, so I thought I'd mention it.

I think I looked at this in the past a bit, I'm sure you've found all this, but there's a few different implementations for DNS resolution (all in https://searchfox.org/firefox-main/source/netwerk/dns): native resolution, DNS-over-HTTP, and maybe another one? I think it also goes across IPC boundaries so there's a bunch of serialization/deserialization logic IIRC.

I would definitely focus on adding the few extra types that Thunderbird needs and not try to create some whole abstract support any type of DNS query.

That's my plan as long as everyone's ok with that.

Comment 6

[Patrick Cloke [:clokep]]

(In reply to Eleanor Dicharry from comment #5)

(In reply to Patrick Cloke [:clokep] from comment #3)

I don't have any other thoughts besides that we use that same code for MX and TXT messages. Not sure if this is just about SRV or not due to the title and description mismatching.

Yes, but we still need the ability to send the MX query in Necko even if it's the same parsing as TXT for the DNS RR, so I thought I'd mention it.

I think the parsing is slightly different for MX and TXT (https://searchfox.org/comm-central/rev/4b7e2607b487a67b9103f607353b0c2ae92003fd/mail/modules/DNS.worker.mjs#223-261), TXT can be made of multiple segments, although how you interpret those segments is left to the calling code. Bug 1583053 fixed this in the DNS.sys.mjs case.

I'm not as familiar with the underlying parsing of MX records.

Comment 7

[Eleanor Dicharry]

(In reply to Patrick Cloke [:clokep] from comment #6)

I think the parsing is slightly different for MX and TXT (https://searchfox.org/comm-central/rev/4b7e2607b487a67b9103f607353b0c2ae92003fd/mail/modules/DNS.worker.mjs#223-261), TXT can be made of multiple segments, although how you interpret those segments is left to the calling code. Bug 1583053 fixed this in the DNS.sys.mjs case.

I'm not as familiar with the underlying parsing of MX records.

Sorry, I just misunderstood your original comment, and yes the parsing is different. I added MX to the description.

Comment 8

[Eleanor Dicharry]

Attachment: WIP: Bug 2000912 - Add DNS MX record support.

Comment 9

[Eleanor Dicharry]

I started putting together a patch to make MX records work in Necko (we need them too, and they are slightly simpler than SRV records) just to understand what migrating TB would take, and after discussing what I found with the team, we decided to try for a smaller fix for our current issue, at least for now. As much as I would like to switch over to Necko's DNS implementation, we would need to implement full native DNS support for TXT, MX, and SRV records in Necko on Windows, Linux, and MacOS. For a patch to work, we'd probably also need to implement the same record types over DoH since the Necko DNS implementation is subject to the TRR prefs. So with all of that, it seems like too much to take on right now, especially since the issue we're having is specifically limited to MacOS.

Comment 10

[Ben Bucksch (:BenB)]

Hey Eleanor, I'm sorry that you are having a hard time with this one. I've also learned the same lesson the hard way. I've spent maybe 1 or 2 full weeks of work on bug 14328, and had a semi-finished patch, but it was never even reviewed by the Necko team.

A few factors to consider:

• This is very difficult to implement, on all platforms. If you want to do that, my old patch for bug 14328 might be a good starting point, but there's a lot of work left. I personally would not spent more time on it.
• Unfortunately, DNS MX lookup is necessary, as fallback for custom domains hosted by large providers. We did that with the HTTPS service, hosted by Thunderbird. If you look at the version history of FetchConfig, you should find it, but it's a trivial HTTPS query with a text response.

Most importantly:

• DNS SRV is inherently unsafe. DNS "poisoning" and spoofing attacks are known since decades, easy to do, and make DNS very easy to attack. There are multiple known attacks that cannot easily be fixed. We intentionally didn't implement DNS SRV in Thunderbird for that reason, back then.
• We also had that discussion at the IETF recently (mailmaint, September, around AutoConfig, PACC etc.), and the discussion came to the conclusion that HTTPS on a well-known URL is safer than DNS SRV. DNS SRV might be officially deprecated at some point not too far in the future.
• DNS at a data center is easier to defend than local DNS. So, the old solution of asking DNS per HTTPS is safer than the current way of using the local DNS resolver.
• One mitigation could be to query multiple sources and compare the results. E.g. ask the Thunderbird HTTPS service and the local DNS server, or the Thunderbird HTTPS service and Cloudflare DoH (DNS over HTTPS). Use the result only if both match, otherwise throw (which will make this method fail and use the other methods).

Comment 11

[Eleanor Dicharry]

I'm moving this one back into the backlog for now. I have my prototype patch to just do MX via DoH attached to this, but we fixed the primary issue that triggered this and have more pressing priorities. I'd still like to do this at some point, but it's a huge lift.