Closed Bug 14328 Opened 25 years ago Closed 5 years ago

DNS: RFC 2782 not supported (SRV records)

Categories

(Core :: Networking, enhancement, P3)

enhancement

Tracking

()

RESOLVED WONTFIX
Tracking Status
blocking2.0 --- -

People

(Reporter: smoerk, Assigned: BenB, NeedInfo)

References

(Depends on 1 open bug, Blocks 2 open bugs)

Details

(Keywords: helpwanted, Whiteboard: [has patch][needs review][necko-backlog])

Attachments

(2 files, 4 obsolete files)

Mozilla doesn't support RFC 2052: A DNS RR for specifying the location of
services (DNS SRV) (see http://www.vix.com/ietf/rfc2052bis.txt).

I think it's a important feature which should be supported by Mozilla.
Assignee: gagan → gordon
Status: NEW → ASSIGNED
Target Milestone: M14
Bulk move of all Networking-Core (to be deleted component) bugs to new
Networking component.
Target Milestone: M14 → M16
Help wanted.
Assignee: gordon → nobody
Severity: normal → enhancement
Status: ASSIGNED → NEW
Keywords: helpwanted
Target Milestone: M16 → M20
See also Internet Draft draft-andrews-http-srv-00
It takes time to check for SRV records. Be careful with features that
can degrade the user experience! I'd like to see a study of SRV's impact
on web-page load times.
It takes time when an HTTP request times out because the server is down or 
overloaded as well! Any testing which attempts to assess impact on "user 
experience" should take into account not only the "best case" scenarios but 
also the load-balancing and failover benefits of SRV. We live, after all, in an 
imperfect world...
Please guys, we need this improvement on our browser of choice.

SRV RRs are very useful; such enhancement is _innovation_. Just take a look at 
the specs.

There is no good reason to not include this (other than allocating some 
development resources).
mass move, v2.
qa to me.
QA Contact: tever → benc
Can we target this (1.0 or future?)
Summary: RFC 2052 (bis) not supported → [RFE] RFC 2052 (bis) not supported
Some corrections to the previous remarks:
 - Browsing is improved: you'd like the web robust like email? SRV do that!
 - Goal of supporting SRV records is support load balanced & backed-up servers.
 - RFC 2052 is superseded by RFC 2782
 - Dealing with SRV records does not take more time, it uses one DNS lookup
 - People are working on library support to automate SRV records
 - Targeting it at v1.0 seems smart; being able to claim robustness is nice ;-)
All in more detail on http://dns.vanrein.org/srv (or send me email and ask)
I am taking ownership of this RFE.  I have been looking for a programming 
project for some time now and this one is very interesting to me.  I'm thinking 
that all protocols that Mozilla supports should use SRV records, i.e. HTTP, 
NNTP, FTP, POP, IMAP, LDAP, SMTP, GOPHER (does anyone still use this?).  I will 
be creating an outline of the plan for comments.
Assignee: nobody → dnewman
Why was I copied on this bug when it was updated? the update was sent to 

njs+netscape@scifi.squawk.com?  Why?

By the way, the RFC is more than six months old - draft RFCs more than
six months old are questionable - I'd contact the authors to check on their
current felings before writing anything.
Nick/others,

RFC 2052, the originally mentioned topic, is indeed over 6 months, but you may
also have noticed my earlier remark that it was replaced with RFC 2782. This one
is on the Standards Track, and STD 1 specifies its status, so rather than making
an RFC look suspicious, I'd like to see a quote from STD 1.
Allow me to demonstrate: it is listed under the heading of
        3.4.  Proposed Standard Protocols
And that is a result of a specific action by the IESG [see RFC 2026, sec 4.1.1],
so it definately makes sense to implement this particular feature.


Hope this helps to take your worries away,

Rick van Rein.
This past weekend I was working on some "proof of concept" code to kind of hash 
out how it is going to work and some things occured to me that maybe others can 
comment on.

If this was a simple ftp client, such as ncftp, then when I initiate a 
connection to domain.org I could do a simple DNS lookup of _ftp._tcp.domain.org 
and plow through the list of SRV records, sorted by priority and picked at 
random by weight within the same priority.  When a connection failes, remove it 
from the list and try again.  Once a successful connection has been established 
the data concerning SRV records could be thrown away.

The web, by contrast, is a completely different paradigm.  Instead of having 
one continuous connection throughout the life of the visit, a seperate 
connection is used for each "transaction".  And each transaction may be 
comprised of multiple connections when a particular page contains multiple 
objects.

This is where my experience with Cisco Local Directors, ArrowPoints and Central 
Dispatch comes in and confuses the issue.  These devices are keeping running 
tabs on the services that they are watching.  For example the Local Director 
will mark a service as bad when it gets a failed connection.  Even though the 
connection would finally make it over to a good server the user experienced a 
delay assuming the server was off the network and had to wait for TCP 
timeouts.  From that point on the LD would direct traffic to services it knew 
were good.  Every once in a while it would try the bad one just for yuks to see 
if it had been fixed.

So assuming we don't want the user to get hung up on a down server every time 
they traverse the service list we need to keep a running tab on sevices we've 
tried in the past.  So does this mean that once we have queried the SRV record 
we keep it cached within mozilla until the TTL runs out?  Or do we make actual 
DNS queries every time and run through the same algorithm to choose a server?  
Or we could keep a running list of servers we've connected to and how 
successful we were at it in a seperate list.  Then we could do seperate SRV 
queries every time and compare what we get back to our running list.  We would 
then try a server marked as down only if we had no other alternatives.

That then brings up the issue of how long do we keep the information around 
for?  I suppose it is no different than a history, but it eats up memory.
Status: NEW → ASSIGNED
> (...)  For example the Local Director 
> will mark a service as bad when it gets a failed connection.  Even though the 
> connection would finally make it over to a good server the user experienced a 
> delay assuming the server was off the network and had to wait for TCP 
> timeouts.  From that point on the LD would direct traffic to services it knew 
> were good. (...)

Only the connection which "detected" the failed service hangs?

> So assuming we don't want the user to get hung up on a down server every time 
> they traverse the service list we need to keep a running tab on sevices we've 
> tried in the past. (...)

I haven't understood why the user will always get hung up on a
failed service. The LD is not going to deliver newer connections
to a healthy server after one failure?
>> (...)  For example the Local Director 
>> will mark a service as bad when it gets a failed connection.  Even 
>> though the connection would finally make it over to a good server 
>> the user experienced a delay assuming the server was off the network 
>> and had to wait for TCP timeouts.  From that point on the LD would 
>> direct traffic to services it knew were good. (...)
>
> Only the connection which "detected" the failed service hangs?

Correct.

>> So assuming we don't want the user to get hung up on a down server
>> every time they traverse the service list we need to keep a running 
>> tab on sevices we've tried in the past. (...)

> I haven't understood why the user will always get hung up on a
> failed service. The LD is not going to deliver newer connections
> to a healthy server after one failure?

That part I was describing was without LDs and so strictly inside of
the browser.  If we requery DNS every time and have to run down the
list servers listed for the service and there is a down server, the
user will get hung up on that down server.  It will eventually time
out and the browser will try the next server, but the user will have
to wait.  Which is why the implementation might want to cache the
DNS data internal to the browser and keep track of which servers are
up and which ones are down.

Or something else I had thought of, but didn't want to mention
because I didn't like it, was to initiate connections to all servers
listed and use the one that responds first.  Or if there are, for the
sake of argument, 20 objects on a particular page and connections
were opened to 4 differant servers, then each connection can serve 5
objects, assuming each server is weighted equally, through the use of
persistant connections.
> That part I was describing was without LDs and so strictly inside of
> the browser.  If we requery DNS every time and have to run down the
> list servers listed for the service and there is a down server, the
> user will get hung up on that down server.  It will eventually time
> out and the browser will try the next server, but the user will have
> to wait.  Which is why the implementation might want to cache the
> DNS data internal to the browser and keep track of which servers are
> up and which ones are down.

So you're talking about timeout on requests for the TCP connections.

I'm now wondering whether such "healthy cache" breaks the SRV behavior
as specified by RFC2782. The thing explicitily states:

"for each address record found, try to connect to the (protocol,
address, service)."

While the cache seems a smart approach, it would be a shame to have
the awesome SRV feature not adhering to the proper standard. How
about to make the "healthy cache" an optional, user-tunable feature
and let it disabled by default?
Changing title because RFC2052bis was superseeded a long time ago by RFC2782
Summary: [RFE] RFC 2052 (bis) not supported → [RFE] RFC 2782 not supported (DNS SRV records)
*** Bug 162798 has been marked as a duplicate of this bug. ***
Summary: [RFE] RFC 2782 not supported (DNS SRV records) → RFC 2782 not supported (DNS SRV records)
I'm disappointed that support for SRV's hasn't made it into Mozilla after three
years.  There are are very useful device and are now being used by a number of
services: Jabber, LDAP, Kerberos, Windows 2000, etc...

The main reason I want SRV RRs is so I can have multiple SSL Apache virtual
hosts with one IP.  Name-based virtual hosts don't work with SSL, but port-based
virtual hosts would :-)

This document answers many of the questions asked above about how to go about
implimenting http with srv:
http://www.globecom.net/ietf/draft/draft-andrews-http-srv-01.html

It is technically expired, but it's the closest thing to a standard I could find.
I have built a working library that supports SRV records in a generic
way.  This library is intended for simple applications that want to
"grab a good connection quickly" so it may be unsuitable for Mozilla,
but at least it is example code, open source so ready for copy/paste.

The library can be found on
  http://libsrv.sf.net
and it works on Linux, OpenBSD, FreeBSD, MacOS X.
Any Mozilla coders out there who would like a quick-n-easy way to
get popular?  ;-)
There are lots of issues related to caching and DNS, but I don't think that
necessarily should be a bottleneck to adding this support.

What you probably need to do is look at implementing this with a pref, and then
checking it in as off by default. You should also carefully research any
dependent changes to existing code, because you have to sell this to the module
owners.

After some bake time to make sure the existence of the feature is not causing
regressions, you can have individual users turn it on and see how it works. 

Don't worry about caching now, it seems like the problems described w/ the
network load distributors would be issues for normal DNS as well.
Summary: RFC 2782 not supported (DNS SRV records) → DNS: RFC 2782 not supported (SRV records)
I'd like to see this implemented so that things like mail server configuration
can be more automatic.
Is David still working on this bug or should this bug's assignment be changed?

Just curious.

Also consider looking at DNS caching discussions from bug 109313
see also bug 173804 (RendezVous/ZeroConf support)
What's the status on this bug?  Is anyone working on it?

It would be a a very useful feature, but the bug seems to be rather old.

Adam
As long as the RFC was 2052, it was an "Experimental" RFC.  RFC 2782 is
Standards Track.  This should mean that the priority is higher.  I guess that I
would like to hear about what the possibilities of supporting this are.  

I did a search on "RFC 2782 Support Library" in google and found
http://www.vovida.org/document/vocaldoc/util/srv_readme.txt. There seems to be
other support code that could be "borrowed" running around as well.
There is more, actually:
This is an lgpl library which can deal with SRV records withoud dependency on
bind - http://faerion.sourceforge.net/doc/rascal/

currently, it can only be downloaded using CVS, but author will release it as a
package in the near future.

the exact function is, possibly, this:
http://faerion.sourceforge.net/doc/rascal/rascal_connect_service.html
There is also ruliwrap:
http://weinholt.se/hacks/

ruliwrap is a wrapper for getaddrinfo() that uses libruli to seamlessly
add SRV record support to programs that use getaddrinfo(3).
*** Bug 293701 has been marked as a duplicate of this bug. ***
At the risk of triggering stupendous wrath, but maybe some real discussion, too,
I want to suggest a dastardly hack in lieu of the DNS SRV support envisioned in
this bug report. It seems clear that:

- DNS SRV records have been around for a long time -- even this bug has been
around for a long time, but nobody seems to be implementing it for HTTP, even
though may other, newer protocols have been supporting the DNS SRV records long
since.  Even Microsoft can't seem to implement this thing.

- A careful look at the name resolution process might offer an explanation of
why the current SRV proposed standard is not getting implemented in the world of
the interactive web.  No matter how you look at it, if you have to query for an
SRV record, and then query for an A record, you have two round trips through the
DNS.  Even if everything is working perfectly, the name resolution process will
take (nominally) twice as long if you have to go through an SRV resolution step.
 You could imagine fixing the problem by allowing IP addresses in SRV records
(this is not what the present standard says, and there are in general good
reasons for that), but even then, it is hard to optimize the process of looking
for records that usually won't be there.  Negative caching is a hard problem. 
(In other words, you have added a full-scale, uncacheable DNS query to every
lousy URL you want to resolve.  A web browser that needs to build a complex page
from several URLs would have a hard time presenting a reasonably responsive
experience to the user even if all servers that need to say "no SRV record here"
are configured properly and don't drop the unknown request on the floor.)

- Doing SRV lookups is hard from a programming perspective.  It goes against the
grain of the facilities provided in the standard C library.  You almost need to
provide an SRV lookup library, and the more libraries you depend on, the less
portable your result.

- Mozilla is the cutting edge of browser development.  If we could find a better
way to do it and write an RFC and maybe get Microsoft on board if it was simple
enough, we would have taken care of 95% of browsers out there and the rest would
take care of themselves.

- DNS SRV is wonderful for load balancing, but it's overkill if you just want to
run on an alternate port.  (If we can consider the space of TCP and UDP ports to
be part of the address space of an IPv4 network, instead of worrying about
exhausting a 32 bit space, we now have 48 bits, which is a fair amount of
breathing room for the time between now and when IPv6 happens.)

I'm going to propose a simple flag that could be inserted in a set of DNS A
records to indicate that the name describes a service instead of a host and
provide the port where the service can be connected to.  It will have the
advantages that:

- It is less than a dozen lines to add (or remove) in only one place,
- It is ridiculously portable, 
- It imposes negligible performance/bandwidth/processing penalty/overhead,
- It solves the problem of naming a service on an alternate port, and
- It is safe in that it will not be triggered unexpectedly (keep reading).

Disadvantages:
- I'm proposing it here for the first time (yes, I know I need to write an RFC),
while DNS SRV has been around forever.  Then again, it might actually be safe
enough to implement without killing performance.
- It does nothing to address the load balancing issue.
- It is supremely crufty (but look how popular NATs are!)

Let me formalize the idea.

The space of loopback addresses comprises a whole /8 (Class A) network but only
contains one address.  However, because of the way it implemented on virtually
all machines, it necessarily gobbles up a full /8 that can never be routed
anywhere on the internet, no matter how much we want the address space back.  If
you want to try some weird stuff, add aliases to 127.0.0.2, to 127.0.0.3, and to
the like to your loopback interface.  You can't do anything with them, and you
can't really tighten up the subnet mask, either.  Therefore, any IP address in
the range 127.0.0.2 to 127.255.255.254 (127.255.255.255 is a loopback
broadcast!) is loony and won't occur as an address to route to on the public
internet except perhaps from machines so grossly misconfigured as to be beyond
the scope of the discussion.

Therefore, it is safe to add A records in the form 127.x.y.z to DNS A record
sets for domain names used only for one service (like www.foo.com) for use as a
flag to indicate special information (in this case, an alternate port) for the
connection.  Machines that don't know what to do with the extra information will
try to connect to the weird loopback address in its turn and quickly discover
that it doesn't go anywhere.  DNS servers that don't provide the extra
information don't waste anybody's time.  And there is practically no chance of
encountering domain name A records that point into this space because they
cannot possibly be useful for anything (that is, unless we define something
useful here).  The bonus is that there is practically no penalty to check for it
(which cannot be said about other solutions).

I propose the following definitions for A records in the form 127.x.y.z:
x = 6 (This is the IP protocol number for TCP.  It can be generalized to 17 for
UDP later, or maybe even something else.  At a minimum, it avoids conflicting
with 127.0.0.1 and 127.255.255.255.)
y = port number divided by 256, discarding remainder
z = remainder of port number divided by 256 (port number modulo 256)

As an example, an A record flag for the name of a webserver that is located on
port 8080 would be 127.6.31.144.  A complete example might be:

www     IN     A     12.23.34.45  ;First IP address on first network
        IN     A     54.43.32.21  ;Second IP address on second network
        IN     A     127.6.31.144 ;Alternate port flag for NAT

This needs no change in DNS server machinery, routers, operating systems,
standard libraries, or anything except the actual application that uses it.  The
actual application (in this case, a web browser) need only have the following
code (here, pseudocode) added in between the call that returns the list of IP
addresses for a domain name and the code that loops through the list of IP
addresses, trying to make a connection:

FOR index = 1 TO A_record_count 
IF host_byte_order(A_record[index]) .BITAND. 0xFFFF0000 .EQUALS. 0x7F060000 THEN
connection_port=network_byte_order(to_16_bits(host_byte_order(A_record[index])))
A_record[index] = A_record[A_record_count]
A_record_count = A_record_count - 1
ENDIF
NEXT index

(Mind my one-based arrays if you convert this to C/C++.)

The hack is utterly heinous, I know, and it does nothing about load balancing
(more flags?), but it should be obvious that this approach is unique in its
ability to allow alternate ports with no additional DNS lookup overhead.  (You
can't say, "no overhead" if you have to go and lookup another kind of DNS RR,
regardless how cool the RR might be.)

I propose this hack, not because I think that it is better than DNS SRV in
general, but because I think that DNS SRV implementation is stalled in the HTTP
sphere due to irresolvable performance problems, and this might be better than
nothing.

(I'm just looking for a bit of discussion at this point.  Implementation, as
above, is trivial, and the above comments could form the basis of an
Experimental RFC.  Such a kludge as this really ought to be discussed, however,
before being unilaterally checked into the source tree, no matter how safe it
might be.)
Bugzilla is not a place for creating new RFCs. sorry. You should try NNTP.
also, You are not right, about need of several DNS requests in process of SRV
queries. DNS answer includes "additional" section which already holds the
required data.
for example:

$ host -a -t SRV _irc._tcp.forestnet.org
Trying "_irc._tcp.forestnet.org"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5356
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;_irc._tcp.forestnet.org.       IN      SRV

;; ANSWER SECTION:
_irc._tcp.forestnet.org. 3600   IN      SRV     1 0 6667 irc.forestnet.org.

;; AUTHORITY SECTION:
forestnet.org.          3600    IN      NS      ns1.cwn.ru.
forestnet.org.          3600    IN      NS      ve208.msm.ru.

;; ADDITIONAL SECTION:
irc.forestnet.org.      3600    IN      A       81.9.48.3
irc.forestnet.org.      3600    IN      A       195.91.162.198
irc.forestnet.org.      3600    IN      A       213.248.60.142
ve208.msm.ru.           3600    IN      A       213.248.60.142

Received 190 bytes from 192.168.0.1#53 in 725 ms


portability is not a big question too. The libraries which were mentioned
earlier are VERY portable.

p.s. just to clear things up. Rascal is BSD-licensed now:
http://faerion.sourceforge.net/librascal.html.var
I have worked with SRV records before (they're used in SIP networks, see
RFC3263), and I can tell you that multiple round-trip is not an issue at all.
SRV and A records will be returned in 1 packet, if there's enough room (that's
where RFC2671 comes in handy).

Note that 127.0.0.1/8 was originally a routable network. I still have customers
that use them, both on private networks and as 'internal' addresses (only
routable in 1 host). But it becomes harder and harder to use them, because lots
of routers and IP-stacks are now hardcoding them (due to RFC3330).
Well, as I'm seeing it right now, we have portable libraries, and we have
emerging technologies like Coral Cache that stand to make excellent use of it,
but no one can use it because the browsers aren't embracing it. :(

Is there really anything left about this whole process that is left to debate,
or is it really a matter of lack of a coder wanting to work on it?  If it is the
latter, then I think we're almost approaching the point of needing to place a
bounty on it.  Truly (affoardable) redundant systems are just dying for this
technology.  You can place a load balancer in your data center all day long, but
if your load balancers fail,  you're down.  If you can implement SRV, then you
fail to another system, and if configured correctly, you can even pull from
Coral Cache.

Anyway, thoughts?  Should be be starting a bounty on this?

Here are some pertinent links:

http://www.ietf.org/rfc/rfc2782.txt

Implementation Examples (not limited to http)
http://www.voip-info.org/tiki-index.php?page=DNS%20SRV
http://www.cisco.com/univercd/cc/td/doc/product/voice/sipproxy/admingd/ver2_1/ddns.htm
I made a Mozillzine forum for people who want to continue non-development discussion:
  http://forums.mozillazine.org/viewtopic.php?p=1676455#1676455
In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
each case, no matter if there are performance questions or anything like this.
The reson is: RFC2782 is a proposed standard according to the IETF.

Are there any current developments regarding SRV RRs in Mozilla products?
(In reply to comment #35)
> In my opinion SRV RRs should be supported by Mozilla/Firefox/Thunderbird/etc. in
> each case, no matter if there are performance questions or anything like this.
> The reson is: RFC2782 is a proposed standard according to the IETF.
> 
> Are there any current developments regarding SRV RRs in Mozilla products?

I don't think there is, however just for grins, some pseudo-code for someone
familiar enough with the zilla sources, here are a pair of methods to give a try:

presuming someone has typed in a plain url of the type http://www.domain.tld,
which of course presumes port 80:

Rather than simply attempting to connect to www.domain.tld on port 80, first
attempt to get _http._tcp.www.domain.tld.  Respect whatever is returned
(although to be honest, I'm not sure what happens at this point.  I really
should try this from telnet sometime to see exactly what occurs here).

Second would be to see if there is a dns library floating around out there that
is lightweight and can handle srv records.  When the user hits enter or clicks
the 'go' button, do an actual dns resolution before proceeding, again presuming
_http._tcp.www.domain.tld.  Respect what is returned.

I believe the crux of the matter is timeouts.  How long do you timeout on trying
srv's?  If you go the second route and do a full name resolution and you get
valid srv data back before then reverting to traditional name resolution, I'd
then proceed with the correct priority site.  Should that time out, then we
would stray from tradidition a bit.  Rather than giving the usual timeout
message, give a new error:

"The requested page could not be retrieved, however the site has specified n
additional backup pages.  Would you like to try the next one now?"
[Please take me to the next one.] [No, Thanks though.]
Don't ask me this again.[x]

Just a couple of thoughts on how to deal with it.  The user experience wouldn't
be damaged in any way.  The srv record timeouts could be adjusted through
about:config, and if done correctly an extension writer could probably extend
things a bit and make them prettier.
I do not think that the retry logic has to be changed, since browsers should
already try all returned A records. SRV just adds ordering to that logic.

I can offer some SRV query code if anyone wants to adapt it mozilla, but it is
quite easy anyway.
*** Bug 319939 has been marked as a duplicate of this bug. ***
indeyets@gmail.com: lgpl code isn't acceptable for mozilla, please don't suggest it.
thomas@gmx.net: if you're really interested, i'd suggest you try doing it, build mozilla, hop on irc and ask for help. offering sample code isn't ilkely to get this very far.

personally, this whole thing seems very annoying. i don't feel like reading the rfc, but i would like a basic specification that explains what to do when:
1. there's no SRV record
2. there are 20 alternates in the SRV record and all of them are bad.
3. general cases in between.

The spec should list the series of requests (dns, http, other) as well as wait points and expected time spent  at each step. the cost (size) of each inbound and outbound request should be listed, and a comparisoon should be made against today's browsing behaviors.

the following browsing behaviors are available today:
1. host doesn't respond, user tries a different link to a different server
2. user tries google cache for link
3. user tries coral cache for link
(In reply to comment #39)
> personally, this whole thing seems very annoying. i don't feel like reading the
> rfc, but i would like a basic specification that explains what to do when:
> 1. there's no SRV record
> 2. there are 20 alternates in the SRV record and all of them are bad.
> 3. general cases in between.

To be frank, "seems very annoying" and "don't feel like reading the rfc" seems like a pretty poor attitude to take.  If you don't want to take the time to form an educated opinion on the bug, no less perform any work on it, that's certainly your right, but if that's going to be the case, why not just leave it alone?

Anyway, to answer these three questions:

1) If there's no SRV record, everything works as it does now, you look up an address record and make the request.

2) This exact case is not addressed in RFC2782 or Mark Andrews' HTTP SRV draft.  I would see two possibilities - either the normal "this request timed out" error is displayed immediately, or a fallback to the standard A record / port 80 behavior takes place.  I think the former would be more "correct", as specification of an SRV record says "if you speak SRV, you should use this".

3) I'm not sure what you mean by general cases in between.

If there are any developers out there interested in implementing this functionality (to be contributed back to Mozilla, if it will be accepted) under contract, please contact me off-bug, as my company is interested in sponsoring this work.
Where should the dns srv code be added?
nspr or necko only?

I would like to add support using OS API calls and avoid dns libs
DnsQuery (win2k+)
res_query (linux, osx)

Is this enough platform reach to add it to mozilla?
(In reply to comment #41)
> Where should the dns srv code be added?
> nspr or necko only?

I'm not sure about that. However, I have been playing with SRV using the POSIX APIs and I'll be willing to contribute the code to Mozilla. I'm not sure if the code will work unchanged under Windows without Cygwin, though it might with MS' pseudo-POSIX libraries.
*** Bug 356150 has been marked as a duplicate of this bug. ***
quote:

Here is a patch that adds SRV support using OS specific api calls
(making use of local cache etc..).
bug 14328

I'm trying to make use of existing interfaces which has the following
concequences:
- the PRNextAddr doesn't take a port or hostname so the only way to
get that info is through PRNextAddrAsString which returns
"hostname:port".
- people advised to to put the code in netwerk/dns so the code could
be easier approved.. but at the end i think the code belongs in the
nspr.

This patch works, but is not really cleaned because of the large
chance of having to refactor it
Attachment #242044 - Flags: review?(cbiesinger)
Given the need elsewhere (see bug 356104) to support not just one, but several other DNS record types, I think it would make sense not to implement the lowest-level part of this with a single routine that can look up (potentially multiple) records of various types, including TXT and NAPTR records. This could also potentially be extended to support other record types in future. 

For example, NAPTR records are essental for DDDS, and TXT records are becoming important in supporting a number of new protocol extensions such as SPF and DomainKeys.

If this was also accessible as an XPCOM service, this would be very useful for adding new functionality from Javascript as well.
> I think it would make sense not to implement the
> lowest-level part of this with a single routine that can look up (potentially
> multiple) records of various types, including TXT and NAPTR records. This could
> also potentially be extended to support other record types in future. 

As the RR spec is designed to be extensible. nsIDnsRecord should be able to hold any type of RR. So we can gradually add more RR types _when needed_. 

> If this was also accessible as an XPCOM service, this would be very useful for
> adding new functionality from Javascript as well.

ResolveSRV (and the async one) make use of nsIDnsRecord and are accessible from js.
Believe me, NAPTR and TXT record support are needed right now.

TXT record support is currently needed for frameworks like SPF and DomainKeys. They would also be useful for a number of more exotic services. 

NAPTR records are needed right now if you're going to implement any servive that uses DDDS; for example ENUM services and SIP.

MX and PTR records would be useful, too: consider things like performing double-reverse-lookups.

If these two record types were also supported, say with resolveTXT, resolveNAPTR, resolveMX and resolvePTR methods, this would also close bug 356104 (at least for the time being).

However, what I was proposing was more generic. By implementing an interface like

nsIIDNService::resolveWithRecordType(name, record_type)

where "record_type" is a string in the obvious format, and returns an iterator in exactly the same way as the proposed resolveSRV method, this would remove the need for a method per record type, thus allowing extension of the repertoire of supported records in future without changing the interface.

(I'm not sure what the behaviour should be if a particular record type was not supported, though; presumably something should be chosen which maximizes the possibilities for forwards and backwards compatibility)

resolveSRV could then be implemented on top of this, without any loss of generality or efficiency.

resolve() can stay to resolve cname and A records (and backwards compat)
a method probably called resolveWithType(...) could implement all other extensions.

Probably we need to extend nsIDnsRecord to hold key/value data (propertybag) of the different RR types (for example weight, priority, type, ip, hostname) or have them as a bunch of typed members.
You can see what kind of members we need looking at the DNS RR types here http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dns/dns/dns_structures.asp
I completely agree: in which case, all records looked up with the new interfaces (even A records) should return property-bags, even if they only return a single field.

If property-bags are too complex, simple lists of strings, or strings and ints, would also be perfectly adequate.

Needless to say, these more complex return values should only apply to the new resolveWithType() interfaces, so that the existing resolve() interfaces remain 100% compatible with existing implementations.
Attachment #242044 - Flags: review?(darin)
Comment on attachment 242044 [details] [diff] [review]
OSX, Windows  and (most probably) Linux using OS API calls

Can you please document the API proposed for nsIDNSService?
(In reply to comment #51)
> (From update of attachment 242044 [details] [diff] [review] [edit])
> Can you please document the API proposed for nsIDNSService?
> 

what about this?
This approach adds 2 methods and an interface per record type.
I added SRV and TXT as examples. The old A record stays untouched.


interface nsIDNSService : nsISupports
{
    /* TXT Record */
    nsIDNSTXTListener resolveTXT(in AUTF8String aHostName);
    
    nsICancelable asyncResolveTXT(in AUTF8String       aHostName,
                                  in nsIDNSTXTListener aListener,
                                  in nsIEventTarget    aListenerTarget);
    /* SRV Record */
    nsIDNSSRVListener resolveSRV(in AUTF8String aHostName,
                                 in AUTF8String aService,
                                 in AUTF8String aProtocol);
                                 
    nsICancelable asyncResolveSRV(in AUTF8String       aHostName,
                                  in AUTF8String       aService,
                                  in AUTF8String       aProtocol,
                                  in nsIDNSSRVListener aListener,
                                  in nsIEventTarget    aListenerTarget);
    /* A Record (unchanged) */
    nsICancelable asyncResolve(in AUTF8String       aHostName,
                               in unsigned long     aFlags,
                               in nsIDNSListener    aListener,
                               in nsIEventTarget    aListenerTarget);
                               
    nsIDNSRecord resolve(in AUTF8String   aHostName,
                         in unsigned long aFlags);
};

interface nsIDNSTXTRecord : nsISupports
{
    readonly attribute ACString text;
    boolean hasMore();
    void rewind();
};

interface nsIDNSSRVRecord : nsISupports
{
    /* weight and priority are just informational because
     * nsIDNSSRVRecord sorts the records beforehand
     */
    readonly attribute long weight;
    readonly attribute long priority;
    readonly attribute ACString host;
    boolean hasMore();
    void rewind();
};
What happened to that patch? What would be needed to get SRV support commited?
Because i filed a related bug ( #393372 ) i just want to ask if anyone was able to get SRV records at all.
I tried DnsQuery on W2K and WXP and asked sites like myspace, youtube, google and microsoft. All of them returned several A records but no SRV records.
Those sites don't publish SRV records. You could try "_http._tcp.butter.eu" which has one.
Thanks. That works (got one SRV record).
But what is the sense in implementing this when no major sites are offering such records? Fit for the future? You have still to check for A records.
comment 56 says "hat is the sense in implementing this when no major sites are offering such records? Fit for the future?"

Well SRV records are clearly a good idea, they are a more generic version of the MX record applying to mail receipt.  The problem is chicken and egg ... you cant expect any sites to support when the browsers dont support it.  So if we start with firefox supporting SRV, then sites will have an interest to provide SRV records, and sites will work more reliably for browsers supporting SRV records, and then IE will want to add SRV record support to catchup with firefox.
Just wait until networks like Akamai start using SRV records to replace their current short-TTL records. Or websites with massive load balancing (Youtube, Gogoel, etc ...)
Thanks. Got it.
i work with a hardware load balancer to balance several web servers and i can tell that i would prefer SRV to do that job and dump that horrible hack that is the load balancer...

but why would i try to configure the SRV records and even worst, tune then, if no users will use it, or even i cant test it with a browser

browsers need to support this first... if its supported, people will use it as its usefull, alot simpler and cheap than load balancers
Boris, Christian, Darin:
any hope to get a review in time for Gecko 1.9 ?
I don't think I'm qualified to review this code.

I also think you should have poked the reviewers about 6 months back.  At this stage, even if it had reviews it might not get accepted -- 1.9 is locking down.
I too would very much like to see this implemented in the very near future.  The sooner the better in my books as I think srv records are way past due on the net and and this 8 year old enhancement request really needs to be implemented.
I have this code running and stable in our mozilla app for a long time now. With new linux/osx/windows code. Although I'm using the GetNextAddrAsString hack as in the current patch.

So I'm just waiting until someone with authority in this area can tell me if the interface I proposed is the right way to go. I'm not going to rewrite our own code to a new interface if no one here cares.
Flags: blocking1.9?
Whiteboard: [has patch][needs review]
We've shipped something like 11 major releases (and over a hundred minor releases) without this, not going to block.
Flags: blocking1.9? → blocking1.9-
Flags: wanted1.9.0.x?
This is entirely inappropriate to take in a security/stability release.
Flags: wanted1.9.0.x?
(In reply to comment #66)
> This is entirely inappropriate to take in a security/stability release.

I thought it was debated that the follow-up 1.9.0.x/1.9.1 could be more than that.
Anyway, 'wanted‑next=?' then?
Attachment #322140 - Flags: review?(cbiesinger)
I have submitted the patch in Attachment #322140 [details] [diff] for adding SRV/NAPTR support.

I have added resolveWithType(...) and asyncResolveWithType(...) API. 
The sync API returns a nsISimpleEnumerator that is used as an iterator to the resolved records that could be of type nsIDNSSRVRecord/nsIDNSNAPTRRecord or nsIDNSRecord.

Blocks: 317491
I have attached a modified patch that includes OSX compilation fix, libxul build compilation fix and a better way for implementing the nsISimpleEnumerator interface for the new listener.
Attachment #322140 - Attachment is obsolete: true
Attachment #322495 - Flags: review?(cbiesinger)
Attachment #322140 - Flags: review?(cbiesinger)
Flags: wanted1.9.2?
Unassigning from current assignee, who did not comment since 2001' comment 15.
Assignee: dnewman → nobody
Status: ASSIGNED → NEW
(In reply to comment #62)
> I don't think I'm qualified to review this code.

(13 months later)

Boris, do you know who could review this ?
You, Christian and Darin are the only ones listed on http://www.mozilla.org/owners.html...
Attachment #322495 - Attachment description: Linux, Windows and OSX SRV/NAPTR support → Linux, Windows and OSX SRV/NAPTR support
Attachment #322495 - Flags: review?(dcamp)
Comment on attachment 322495 [details] [diff] [review]
Linux, Windows and OSX SRV/NAPTR support

Dave, Boris suggested me to try asking you...
Is this patch going to make it to Stable release anytime soon?  I am no programmer and have no clue what to do with this patch.  I would like my web sites to have backup/load balancing features without the need to rely on hardware devices that can break down/may be off the Internet cause of a route failure.
QA Contact: benc → networking
Attachment #322495 - Flags: review?(jduell.mcbugs)
today: 10th anniversary of the bug. let's party!
RFC2502bis referenced in the description has now been published:
http://tools.ietf.org/html/rfc2782

Some internet-drafts related to SRV and DNS records:
as mentioned above (expired):
http://tools.ietf.org/html/draft-andrews-http-srv

for determining transport - HTTP has been run over SCTP and TCP, IPv4 and v6; how would you know which to use? Possible methods include DNS record lookups and specifying the transport direct in the URI.
http://tools.ietf.org/html/draft-natarajan-http-over-sctp
http://tools.ietf.org/html/draft-yourtchenko-tran-announce-dns
http://tools.ietf.org/html/draft-wing-http-new-tech
http://tools.ietf.org/html/draft-wood-tae-specifying-uri-transports
http://tools.ietf.org/html/draft-jennings-http-srv
http://tools.ietf.org/html/draft-faltstrom-uri
(In reply to comment #76)
> RFC2502bis referenced in the description has now been published:
> http://tools.ietf.org/html/rfc2782

it's already in the title :-)
 
> Some internet-drafts related to SRV and DNS records:
> as mentioned above (expired):
> http://tools.ietf.org/html/draft-andrews-http-srv
> 
> for determining transport - HTTP has been run over SCTP and TCP, IPv4 and v6;
> how would you know which to use? Possible methods include DNS record lookups
> and specifying the transport direct in the URI.
> http://tools.ietf.org/html/draft-natarajan-http-over-sctp
> http://tools.ietf.org/html/draft-yourtchenko-tran-announce-dns
> http://tools.ietf.org/html/draft-wing-http-new-tech
> http://tools.ietf.org/html/draft-wood-tae-specifying-uri-transports
> http://tools.ietf.org/html/draft-jennings-http-srv
> http://tools.ietf.org/html/draft-faltstrom-uri

Thanks for the links. I have skimmed through the drafts and I don't see any advantage over SRV / rfc2782. Things like http+srv are a usability nightmare, others trying to implement something similar, but with less flexibility. rfc2782 is a simple and good standard.

I still wonder, why nobody is pushing this for firefox.
This still give us no clue who are qualified to review patch, while owners pages suggest its Christian Biesinger, Darin Fisher, Boris Zbarsky.
Another poor soul pleading for updates on this bug. Given the context (a *ten year old* bug) I would personally argue that this is not even bad form anymore.

Please, Mozilla, please fix this. This is the one thing preventing people from being able to set up cheap hosting easily. The only functionally equivalent alternative is lots of heartbeat hosts and very low TTL DNS records, which is a strain on both administration of those hosts and on the DNS system.

Everybody will benefit from this. I beg of you, somebody, please at least tell us what is going on here. Why is nobody working on it?

To put my money where my mouth is: I am willing to actually pay money to whoever decides to implement this. Not much, but if a few other people chip in I bet we could make at least a modest sum. It will definitely save me the hosting costs, so I think it would be worth it.
Hraban, the reason this bug is unresolved is that there is no standard for HTTP in SRV.  someone needs to write an RFC and get it approved.  Mark Andrews wrote a draft, but it never reached RFC status.

the SRV spec says that a standard must explicitly support SRV, you can't just apply it to a random protocol.  it also says that the name from IANA assigned numbers should be used, but the registration for port 80 specifies several names: "http", "www" and "www-http".  in other words, it's not clear if you should query for _http._tcp or _www._tcp -- this is why a formal specification is needed.

IETF is open for anyone to submit a draft.  it just takes some time and interest for writing formal documents and engaging in IETF discussions.
Kjetil,

You are bringer of terrible news. I hate to say that this seems to be right... before we can use this:

1) The SRV draft must become a real standard.
2) The HTTP standard must explicitly specify how to use it.
3) Some browser must support it.

I thought step 3 alone was daunting enough, but this... What a shame.

Regardless, thank you for your very clarifying post. I think I can finally move on now. ;)

Greetings,

Hraban Luyat
It appears clear that 'http'  is the preferred service name for an interactive browser such as Firefox  that is used for human-readable content. 
As listed:

DNS-SD Service types
http://www.dns-sd.org/ServiceTypes.html
Blocks: 536915
I'd also like to see it, specifically with XPCOM access for extension use and use in other protocols like XMPP...
Depends on: 545866
The patch here is great, because it shows how to make arbitrary DNS queries, but it unfortunately limits the API to SRV and TXT. In bug 342242, we need MX, too. So, I filed bug 545866 for an IDL API to allow arbitrary DNS record types to be queried from within Mozilla. This bug could then use this API.
Mozilla Firefox is a brilliant piece of software, the programmers and contributors have many reasons to be extremely proud of it.  The number of people that I have shown Firefox to must be in the hundreds now and many of them continually remind me how much they appreciate it and are so glad that I showed them how to switch from Internet Explorer.

I would like to say that this little bug, which is very old now, is now more important then ever to the Internet and I am sure millions of sys admins and other IT professionals would dearly appreciate its inclusion in Firefox and the other superb software projects of Mozilla.  Please can I add a hazaa to the developer that finally gets this included and lets hope it is before the end of this year.  Lets make it a date, inclusion before the end of the year :).

Regards to all.
blocking2.0: --- → ?
Flags: wanted1.9.2?
Just to add my voice to this bug. There is a patch. It's a ten year old bug.
So what ? ... Still not implemented in main release.
Not blocking the release on this, we'll release this when it's ready, but not holding a release for it.
blocking2.0: ? → -
(In reply to comment #87)
> Not blocking the release on this, we'll release this when it's ready, but not
> holding a release for it.

In other words:

> no
joining the godot queue
Indeed the relevant draft is here:

http://www.anta.net/nic/draft-andrews-http-srv-01.shtml

I'm going to do some research to find out why it stalled.

SRV will be so useful it is stunning that it has languished for so long.
Attachment #322495 - Flags: review?(dcamp) → review?(dwitte)
Attachment #242044 - Attachment description: OSX, Windows and (most probably) Linux using OS API calls → OSX, Windows and (most probably) Linux using OS API calls
Attachment #242044 - Flags: review?(darin.moz) → review?(dwitte)
Please also see the patches in bug 545866, which are an alternative to this.
Someone who's more familiar with DNS and the relevant RFC should probably review this...
     Hello, I host a web server from home. ISP's, to protect that insecure OS 'broken' windows, block the standard lower ports. I use my server for private use such as ssh. This server could also be classified as a LAMP server. I would like the HTTP and HTTPS services to work without the user having to either see or type in the high port numbers for these services, 10080 and 10443. I only recently learned of SRV records and what they offer to someone like me, and what they offer to services that have redundant backup systems.

     SRV records sure do solve a ton of technical problems. However, for some reason, browsers only look up the A record, and this record does not say anything about the services on the server
     Hello, I host a web server from home. ISP's, to protect that insecure OS 'broken' windows, block the standard lower ports. I use my server for private use such as ssh. This server could also be classified as a LAMP server. I would like the HTTP and HTTPS services to work without the user having to either see or type in the high port numbers for these services, 10080 and 10443. I only recently learned of SRV records and what they offer to someone like me, and what they offer to services that have redundant backup systems.

     SRV records sure do solve a ton of technical problems. However, for some reason that I cannot comprehend, browsers only look up the A record, and this record does not say anything about the services on the server or what port they are running on.

     This bug was submitted in 1999! Really? This was a serious bug in 1999, now it's 2011! What happened? Over ten years and no solution. That is said. This would be such an improvement for Firefox, so many people would use it. I see complaints on blogs all over that their ISP blocks lower ports and they have to use a lower port and express it in the URL. It is a serious flaw, just search for it. I just seems stupid. You should be able to express the port in the URL, but why can't the browser just figure it out from a SRV record when it's not defined in the url, and use default ports when its not defined as a SRV record either? Seems like that would not be a hard thing to accomplish in a month or two, but its been OVER TEN YEARS!!! I just hope it doesn't end up being another ten years. Maybe the next release of firefox?
(In reply to comment #94)
>      This bug was submitted in 1999! Really? This was a serious bug in 1999,
> now it's 2011! What happened? Over ten years and no solution.

this is not solely due to Mozilla -- the IETF has been dragging its feet, too.  there is still no RFC which specifies *what* SRV tag to use for HTTP (there are three candidates, "_www._tcp", "_www-http._tcp" and "_http._tcp".  the current status is that we're waiting for http://tools.ietf.org/html/draft-ietf-tsvwg-iana-ports-10 to be passed as a new RFC, which should happen very soon.  (according to the document, "_http._tcp" should be used as the SRV tag for HTTP, so there won't be a big surprise.)
Comment on attachment 322495 [details] [diff] [review]
Linux, Windows and OSX SRV/NAPTR support

Assigning review to Patrick, who knows DNS best at this point.
Attachment #322495 - Flags: review?(mcmanus)
Attachment #322495 - Flags: review?(jduell.mcbugs)
Attachment #322495 - Flags: review?(dwitte)
Attachment #322495 - Flags: review?(cbiesinger)
Attachment #242044 - Flags: review?(mcmanus)
Attachment #242044 - Flags: review?(dwitte)
Attachment #242044 - Flags: review?(cbiesinger)
Attachment #242044 - Attachment is obsolete: true
Attachment #242044 - Flags: review?(mcmanus)
Comment on attachment 322495 [details] [diff] [review]
Linux, Windows and OSX SRV/NAPTR support

First, thanks for laying this groundwork. I'm not convinced that we should be doing http LB via SRV all the time, but certainly the platform should have the notion of access to SRV records and this approach is sensible enough.

I've only looked at this very briefly, but I just want to confirm my understanding that this adds new interfaces (an their implementations) for looking up some additional RR types... but it does not actually make any existing code (e.g. the http connection code via the socket service) use this code. Correct?

These lookups are also uncached (by mozilla at least) - correct?

I'm not yet passing judgment on either of those things - just seeking confirmation as I've only skimmed the patch.

Also, if I was prepared to try and move this code forward, perhaps with some required additions and integration, would you be prepared to update the patches both for the bitrot they no doubt have suffered and for those changes? It is completely fair to assume you might have moved on after the long latency of the bug sitting here. I've got a couple other things queued on my plate - so you should wait for a heads up before assuming I'm ready.
My offer still stands.

€25 to the charity of choice of anybody that makes the dev version of Firefox fall back to SRV out of the box by the end of July, 2011. One extra euro for every person that chips in at least €/$ 10!

(max +10 :P a limit I would be happy to reach)

Cheers!
(In reply to comment #98)

I'm in with 25€ and 1€ für for every other person with $10 or more.
Me too. + € 25,--
(In reply to Kjetil Torgrim Homme from comment #95)
> this is not solely due to Mozilla -- the IETF has been dragging its feet,
> too.  there is still no RFC which specifies *what* SRV tag to use for HTTP
> (there are three candidates, "_www._tcp", "_www-http._tcp" and "_http._tcp".
> the current status is that we're waiting for
> http://tools.ietf.org/html/draft-ietf-tsvwg-iana-ports-10 to be passed as a
> new RFC, which should happen very soon.  (according to the document,
> "_http._tcp" should be used as the SRV tag for HTTP, so there won't be a big
> surprise.)

The IETF draft was published as RFC 6335 last month. "_http._tcp" is the SRV tag that was chosen.
Attached patch Attachment 322495, unbitrotted (obsolete) — Splinter Review
(In reply to Patrick McManus from comment #97)

> I've only looked at this very briefly, but I just want to confirm my
> understanding that this adds new interfaces (an their implementations) for
> looking up some additional RR types... but it does not actually make any
> existing code (e.g. the http connection code via the socket service) use
> this code. Correct?
> 
> These lookups are also uncached (by mozilla at least) - correct?

Correct (as far as I understand at least!)

> Also, if I was prepared to try and move this code forward, perhaps with some
> required additions and integration, would you be prepared to update the
> patches both for the bitrot they no doubt have suffered and for those
> changes?

I would like to see support for DNS queries added in the platform, but I don't have much time to work on it. I've unbitrotted the patch, but I haven't fully read it to do that. If the next round of review comments request small improvements, I would be happy to make them. If they require almost rewriting it from scratch, I may not be able to find time for it soon...

For now I've tried this only on Linux, using this code in the error console:
Components.classes["@mozilla.org/network/dns-service;1"].getService(Components.interfaces.nsIDNSService).resolveWithType("_xmpp-client._tcp.gmail.com", 2, 0).getNext().QueryInterface(Components.interfaces.nsIDNSSRVRecord).host;
Attachment #322495 - Attachment is obsolete: true
Attachment #577170 - Flags: review?(mcmanus)
Attachment #322495 - Flags: review?(mcmanus)
Please note that there are "competing" patches in bug 545866.
(In reply to Ben Bucksch (:BenB) from comment #103)
> Please note that there are "competing" patches in bug 545866.

I saw that bug too, but I don't think the presence of an incomplete WIP (that is over a year old already) in a different bug should stop moving forward with this patch which already reached the review stage in the past.

The changes here also look more contained, so I'm less scared of breaking things in an area of code that I don't know much about.

If you feel your approach in that other bug is better, I don't think moving forward here prevents you from proposing some refactoring later.
Florian, the patch you updated is 3,5 years old, so that can hardly be an argument.
The problem with the patch here is that it's for SRV only, while we need MX, too. The API that it uses it not future-compatible and IMHO a dead-end that shouldn't be pursuit.
> The API that it uses it not future-compatible

Correction: "The API that it introduces is not future-compatible"
(In reply to Ben Bucksch (:BenB) from comment #105)
> Florian, the patch you updated is 3,5 years old, so that can hardly be an
> argument.

The patch worked 3 years ago, and still does (I just had to update it for the new locking API, in addition to the usual trivial bitrot that happen to old patches).

Your patch has never reached completion. I don't see what shouldn't be used as an argument, sorry.

> The problem with the patch here is that it's for SRV only, while we need MX,
> too. The API that it uses is not future-compatible

The resolveWithType method takes an aRecordType parameter. How is that an SRV only API? I don't see how that would prevent adding support for MX records in the future.

> a dead-end that shouldn't be pursuit.

I don't see why you being interested in querying MX records (probably for very good reasons, I can totally see why that would be useful) would mean I shouldn't pursuit querying the SRV records that I need.

If you think this is a dead-end, fine. But please refrain from adding more stop energy comments. Thanks :)
> stop energy comments

Please. I was merely pointing out that I put in almost *a week of work* in *improving* the patch here and making it more general with an IMHO better IDL API and adding MX support.

I just want to make sure that my patch is considered before any new API is accepted in the tree. You can't call that "stop energy", that's insulting.

As far as "not working", I don't think you have actually tried it, have you?

> why I shouldn't pursuit querying the SRV records that I need.

Because the patch introduces a new DNS query IDL API, and I think that API is suboptimal.

> you being interested in querying MX records (probably for very good reasons,
> I can totally see why that would be useful)

Just in case you are being ironic: Yes, we do need this for Thunderbird.

To stop this cat-fighting, let's leave this to the reviewer and super-reviewer, indeed.
(In reply to Ben Bucksch (:BenB) from comment #108)
> Just in case you are being ironic: Yes, we do need this for Thunderbird.
> 
> To stop this cat-fighting, let's leave this to the reviewer and
> super-reviewer, indeed.

Patrick: I know that support for querying MX records is important for Thunderbird, to help improve various aspects of auto-configuration. If it were up to me, I would prioritize MX support above SRV support, and/or prefer a solution that provides both. (I don't have anything against SRV support, but it unlikely to be enabled in Firefox or Thunderbird anytime soon, since compatibility, performance, and security risk analysis that has to be done first.)
(In reply to Ben Bucksch (:BenB) from comment #108)

> making it more general with an IMHO better
> IDL API and adding MX support.

The API your patch proposes may be better, but I haven't looked at both enough to have a strong opinion either way. I still don't understand why you claim the API here isn't future compatible, when both the API proposed here and yours use a type parameter for a resolveWithType method, and a specific idl interface for each record type. The only difference I see is that you are using a string for the type parameter whereas the patch here uses a set of numeric constants.


> As far as "not working", I don't think you have actually tried it, have you?

Right, I haven't tried it, because I trusted your comment (bug 545866 comment 7) saying it wasn't ready. You also hadn't given any clear answer to comments 6 and 9 asking for what needed to be done before it could be ready.

I see you have just added a comment there on this topic, and renamed the attachment from WIP to Fix, although the 20 or so printf calls in the code still make it look like a WIP to me (admittedly if that's the only detail to address it's trivial though :)).

> > why I shouldn't pursuit querying the SRV records that I need.
> 
> Because the patch introduces a new DNS query IDL API, and I think that API
> is suboptimal.

Is there an important difference of the two APIs that I've missed, beside using string or numerical constants for the query types?
The current API returns an nsIDNSRecord, which returns IP addresses in a kind of custom iterator - it has a hasMore(), but it's not an nsISimpleEnumerator.

The patch here just adds another API, with new call in nsIDNSService, new listener nsIDNSIterListener, and new result interface nsIDNSSRVRecord, to iterate over DNS record results, but using nsISimpleEnumerator. So, there are 2 different interfaces to iterate over DNS results, the old and the new one, both being similar, yet separate. Also, the result interfaces nsIDNSSRVRecord and nsIDNSNAPTRRecord are not related, both are directly nsISupports, but both coming from the same nsIDNSIterListener.

In my patch, these APIs are united, so the old API is slightly modified to use nsISimpleEnumerator, and to support all DNS record types in IDL. These are also represented in their own type (there was no other way), but at least they all (including DNS A) share a common base type for results.
(In reply to Brian Smith (:bsmith) from comment #109)
> (In reply to Ben Bucksch (:BenB) from comment #108)
> > Just in case you are being ironic: Yes, we do need this for Thunderbird.

> Patrick: I know that support for querying MX records is important for
> Thunderbird, to help improve various aspects of auto-configuration.

Brian, querying MX records could have been useful in several occasions for Thunderbird in the past, and I'm sure it will still be useful in the future.

I'm however unaware of anybody in the Thunderbird team currently working on a feature that depends on this.

> I would prioritize MX support above SRV support, and/or
> prefer a solution that provides both.

As far as I understand, the patch proposed here doesn't make it harder to implement MX support in another patch.

> I don't have anything against SRV
> support, but it unlikely to be enabled in Firefox or Thunderbird anytime
> soon

After reading comment 97, I thought I didn't need to elaborate on why querying SRV records is needed, but apparently I was wrong on that :).

I'm currently implementing the XMPP instant messaging protocol in JavaScript for Thunderbird as part of https://wiki.mozilla.org/Features/Thunderbird/Instant_messaging_in_Thunderbird

Using SRV records is a key aspect of the XMPP protocol. XUL applications implementing XMPP (http://oneteam.im/, http://instantbird.com, and probably more) have had to ship binary XPCOM components to support this for years, but I would be sad to do it this way for Thunderbird as it really feels like something that should be available in the platform.

> since compatibility, performance, and security risk analysis that has
> to be done first.)

The patch proposed here doesn't change any existing API or behavior, so I don't see how compatibility or performance could be affected.
> I'm however unaware of anybody in the Thunderbird team currently working on a
> feature that depends on this.

Please see the dependencies. Bug 563958 is listed, and it states:

"We need to be able to do DNS MX lookups for that. We don't have that ability in Mozilla yet, that's what bug 545866 is about. For the mean-time, we used a webservice to do the DNS lookup, which is of course silly, but was the only alternative."

That's what Thunderbird still does, but it's so silly and awkward that we'd like to do it properly.

> Using SRV records is a key aspect of the XMPP protocol.

FWIW: Having implemented XMPP in a Firefox extension myself, I understand that and crave for this as much as you do.

> The patch proposed here doesn't change any existing API or behavior

And that's the problem, precisely.

To illustrate my comment 111:
If you need to extend something (e.g. an office building) and find it has no provision to do that, you can either build something different next to it and let them live in parallel, with the resulting gab being a nuisance for anybody using it, or you can modify the existing part and new part as a coherent whole that is nice to use. I tried to do that, with as little modification to the existing part as possible, but yet making a coherent new whole.

Usually, these two approaches are not compatible: You can't first temporarily add the new building and then tear it down again (affecting all apps using this new API), that makes no sense.

Again, I just ask that the reviewers also consider my patch seriously before allowing this more hacky patch to go in.
(In reply to Brian Smith (:bsmith) from comment #109)
> I know that support for querying MX records is important for
> Thunderbird, to help improve various aspects of auto-configuration.

And features like bug 92167 and others.
This kind of dispute between two approaches is something I would look to Christian's module leadership on. cc'd.

Steve Workman has been doing the DNS work lately. cc'd.

I would think we would have an absolute requirement for something that supports srv (because it often part of new protocol discussions and is in some existing interesting ones), mx, txt (probably), and A/AAAA.

IMO they should be tied together in a generic record type which MUST support TTLs and that should be used in the cache layer - that is a major shortcoming of the current implementation. I don't believe either patch addresses that.

I personally find the patch in 545866 a bit more to my liking for a starting point, but they have a lot in common. Others are likely to disagree - but to me it is less invasive to nsdnsservice2.cpp and I appreciate that it integrates the new types into the existing resolver code.

however, 545866 appears incomplete (known bug and full of debug code) and has not had any substantial movemement over recent time - it doesn't make sense to block this functionality on that bug indefinitely; but if the author of the patch in 14328 can't find the time to make the integration similar to 545866 then it might have to wait until someone can find the time - but that's my opinion and I think I should defer to Christian on it overall.
Another use for SRV is for service discovery for CalDAV and CardDAV servers.  I understand that the IETF is firmly behind the standard and that *any* new protocols can be expected to use this for service discovery.

Clearly the person who filed this bug back in 1999 was a visionary!
Comment on attachment 577170 [details] [diff] [review]
Attachment 322495 [details] [diff], unbitrotted

Florian, Ben,

Please engage biesi, the necko module owner, to arbitrate between the approach here or in in 545866 (comment 115). Also maybe talk to steve to see what he has planned. When that's resolved mark r? again and I'll look at the code or work with Steve to see that it happens. this is good stuff we want imo.
Attachment #577170 - Flags: review?(mcmanus)
All, thanks for the patches - sorry it's taken time to get round to looking at them.  FYI, I'm commenting in this bug in response to the work done here and in bug 545866, but that doesn't mean I favor this bug's approach :) I'm just not going to repeat the same comment in two bugs.  I've quickly looked over the work done in both bugs and it ties in nicely with work we're hoping to do to get support for TTL, DNSSec and different record types as you guys worked on.  Like I said, I've quickly looked over these both and haven't made a decision yet, but I'd like to unify the approaches somehow.  Our current plan is to really attack this in Q2, but if possible, it'd be great to agree on a base approach that will require fewer changes later for TTL and DNSSec.  So, some initial thoughts:
-- I'd like to look at caching these results, possibly by integrating with the current cache, although that might mean serious changes. At the very least, I want to see if it's useful to cache them, and if so, then a plan should be in place - it could be ok for the short term to directly query the DNS server each time, as long as a cache implementation is considered.
-- Async requests - I'm not comfortable with creating a thread for each async request.  Again, I'd like to see if we can leverage the existing thread pool for this.  Some issues with this: might require a change for A and AAAA requests to use DNSQuery/res_query; either change the cache and queue structures to deal with other record types, or build a different queue/cache structure
-- res_query is not currently exported on Android.  We ship a copy of the Android stub resolver so we could change this.
-- For DNSSec, further changes are needed.  There is some support there already for Windows, and alternatives need to be explored for Android and the other OSes.
I'll be looking into this more in the next month or so, and hope we can work on a patch that provides for MX, SRV and NAPTR, and more in the future.
> I'd like to unify the approaches somehow.

FWIW, the patches in bug 545866 are based on the earlier patches here.
No longer depends on: 356104
I finally got around to taking a look at the patches here. My apologies for the delay.

I agree with Steve's comments, especially it'd be good to reuse the existing threading infrastructure. Not sure caching is so important because these queries should be rare, right?

I'd really like to reuse nsIDNSListener/nsIDNSRecord, and not having to change all API users would be great as well. Seems like the simplest fix would be to add a "type" field to nsIDNSRecord, and then either add the properties for the various other types onto nsIDNSRecord, or let people QI it to new interfaces.

How does that sound?
> Seems like the simplest fix would be to add a "type" field to nsIDNSRecord,
> and then either add the properties for the various other types onto nsIDNSRecord,
> or let people QI it to new interfaces.

I don't see how you could do that cleanly, given that the existing interface is all about IP addresses, and SRV and TXT aren't.

http://mxr.mozilla.org/comm-central/source/mozilla/netwerk/dns/nsIDNSRecord.idl

> not having to change all API users would be great as well

FWIW, my patch does change them all (those in-tree). There were not many callers, 5 or so. I thought that was acceptable. It didn't appear to me that this API is widely used outside the tree either.

Given the fake non-standard enumerator, and the inability to cleanly extend the nsIDNSRecord, and the few API users, and I think it's best to adapt them to the new situation. As I said in comment 111, I tried to keep the changes to the existing API minimal while still arriving at a clean outcome.
I read a lot what was said here. I believe that Florian has a point though. If I'm reading him well, he is not saying that this patch should be pushed for life.

If I understand well the problematic, it seems this patch is working but (referring to Ben) might add some unecessary and crappy elements that are not needed. He might well be damn right about this.

But, it also seems that both APIs are very similar (the target one and this one I mean). If I'm reading well, flo suggests we read this patch, push it and then consider merging both APIs cleanly later. Because they are pretty much identical it won't be a big hurdle for people updating to update their code to go to the new API.

But doing this is a clear win : it makes SRV available right now (it's been waiting a long time and reading this bug, it might be waiting very long before anything happens if it stays like this). Then, we will have all the time in the world to patch this and do something a lot more like what Ben suggests and that sounds a lot better.

And we can then think about adding better async request using the actual thread pool on this single API.

... And caching if needed ! Maybe it would be possible to consider caching as a whole for every type like this. Actual and future ones.

I really believe we should not consider this as a single problem : it is making this harder than it should be. We might be better off dividing this into separate issues to ease the work. Don't you think ?
Blocks: 735215
This is just painful to watch, after 8 years watching this - come on mozilla!  This bug was opened in 1999!

Even the execrable micro$oft internet exploder on an excuse for an OS I would never run can out innovate this stall-cluster-f.

Like I said in 2004 SRV records are an important improvement to the routeability of HTTP.  Get with the program - lead, innovate, get there first!  Maybe there should be a program that after 5 years a bug is either closed or elevated to P1 to avoid low priority bug progress stall.  

When I was doing software management of a network server component I used to insist that several nice-to-haves were implemented in each release to keep the software quality and feature set progressing.  As a result it was the most reliable product of dozens in the company and never crashed once in a year+.


A software product with only P1s fixed turns ugly IMNSHO.

So fer chrisakes implement it already and move on!!!!

I dont want to be receiving these emails in another 8 years in 2020 :(
(In reply to Adam Back from comment #123)

This is an inappropriate comment for bugzilla, please take it to the forums or newsgroups. This is not how you vote or try to convince people something is important or worth doing. If you are able to help we'd welcome your participation, but either way please read the bugzilla etiquette guidelines before posting again:

https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
Blocks: 787369
Regarding Comment 120, let me suggest that doing caching would advisable.  There is talk of using SRV records for HTTP 2.0.  I don't know if this is a good idea– I have serious concerns.  I also don't know how serious such discussion is.  But I would hate for someone to work on a lot of code, ignore caching, and then see 2.0 require SRV, thus requiring another rewrite.
No longer blocks: 735215
Blocks: 861622
I have some code that does DNS lookups using the OS resolver via js-ctypes. Windows, Linux and Mac is supported. Its not cleaned up yet and certainly not an official solution for this bug, but if you need it please email me and I will send it to you.
Patch from bug 545866

This surely bitrotted in the 4 years it's been ignored, but should implement 90% of what's needed here. I've invested a huge amount of time into it already.

Asking for review, to move this forward.
Attachment #577170 - Attachment is obsolete: true
Attachment #8536761 - Flags: review?(cbiesinger)
(In reply to Ben Bucksch (:BenB) from comment #130)
> Asking for review, to move this forward.

I recommend picking a different reviewer. biesi hasn't worked on necko code in a very long time.

https://wiki.mozilla.org/Modules/Core#Necko
Attachment #8536761 - Flags: review?(cbiesinger) → review?(mcmanus)
Attachment #8536761 - Flags: review?(hurley)
Comment on attachment 8536761 [details] [diff] [review]
dns-545866-6.diff

Review of attachment 8536761 [details] [diff] [review]:
-----------------------------------------------------------------

patch does not apply
Attachment #8536761 - Flags: review?(mcmanus)
Attachment #8536761 - Flags: review?(hurley)
Attachment #8536761 - Flags: review-
Comment on attachment 8536761 [details] [diff] [review]
dns-545866-6.diff

Yes, I said that it bitrotted. It has been ignored for 4 years.

I'd like to know what needs to be done to make this acceptable,
aside from un-bitrotting and making tests pass, of course.
Attachment #8536761 - Flags: review- → review?(mcmanus)
Comment on attachment 8536761 [details] [diff] [review]
dns-545866-6.diff

Review of attachment 8536761 [details] [diff] [review]:
-----------------------------------------------------------------

so ben - thanks for your efforts here.

For anyone reading along at home, please realize that this patch does not change any firefox behavior regarding srv (especially wrt http) - it adds a platform api for accessing specific non-a/aaaa record types. I think this is a reasonable approach and I would take supported complete patch for that. Here are some things I noticed after spending an hour with the patch - please be aware that it may not be exhaustive - I can't do a real review of a patch that won't apply. Lots of it is trivial.

* bitrot.. the first file has even moved
* littered with printfs
* it needs to use the dns cache and deal with query aggregation like the a record code does
* it needs to respsect the threadpool that dns establishes
* the patch has some inconsistent documentation about what types it supports.. I would think txt srv and mx are required.. tlsa, ptr, naptr maybe required too
* nit of 2 space indents in new
* I don't like how nsidnslistener is changed even though asyncresolve remains the same. This is a pretty heavily used interface and will cause lots of breakage. Can we add nsidnstypelistener as the callback to asyncresolvetype ?
* does this work on all tier 1 platforms? (it looks like it might)
* contributors dont add their emails to files anymore
* you have a test that calls to an external host - that won't work

::: netwerk/test/unit/test_dns_other_requests.js
@@ +34,5 @@
> +        do_check_true(record instanceof Components.interfaces.nsIDNSMXRecord);
> +        dump("MX preference " + record.preference + "\n");
> +        dump("MX mail server " + record.mailServer + "\n");
> +      }
> +      else if (record.type == "MX")

srv

::: testing/xpcshell/head.js
@@ -248,5 @@
>  }
>  
>  function do_throw(text, stack) {
>    if (!stack)
> -    stack = Components.stack.caller;

?
Attachment #8536761 - Flags: review?(mcmanus)
Hey Patrick,

thanks a lot for taking the time to review the patch! This is exactly what I was looking for:
a) an opinion on whether the API approach taken is reasonable -
   you seem to agree that this is a good way forward, and I'm relieved about that.
   I spent a lot of time making a reasonable and clean API that's not hotchpotch.
b) a specific list of things to fix
Thanks for that.

> For anyone reading along at home, please realize that this patch does not
> change any firefox behavior regarding srv (especially wrt http) - it adds a
> platform api for accessing specific non-a/aaaa record types

I misunderstood this at first, so I'll clarify for anyone following, esp. patch authors:
* This patch *does* add support (API and code, on Win, OS X and Linux/Unix platforms)
  to query DNS SRV and DNS MX records from Mozilla code, including from
  privileged JavaScript code.
  Any application code that wants to use DNS SRV or DNS MX can do so, with this patch.
* This patch does *not* add any code that actually queries DNS SRV records and uses them
  in Firefox to resolve website servers or anything else.
  This is being left to Firefox, in another bug, because there are entirely different
  considerations required.
  It would also allow e.g. Thunderbird use DNS SRV to properly resolve XMPP servers
  (bug 787369) and be standard-compliant, which is currently impossible, but would be
  a fairly trivial change (maybe 10-20 lines) with this patch.

> I don't like how nsidnslistener is changed even though asyncresolve remains
> the same. This is a pretty heavily used interface and will cause lots of
> breakage. Can we add nsidnstypelistener as the callback to asyncresolvetype ?

I agree. I didn't make this particular API change exactly for that reason, that it would
break too much. I found this a reasonable tradeoff, to leave one deprecated function in,
to not break thousands of lines of code and quite possibly addons. This could be fixed bit by bit.

> * it needs to use the dns cache and deal with query aggregation like the a record code does
> * it needs to respsect the threadpool that dns establishes

Yeah. That part was totally over my head. Somebody else would have to take this bite.

Thanks a lot, Patrick, for the review! I now feel that my extensive time with this patch was not wasted, even if I cannot finish it myself.
Whiteboard: [has patch][needs review] → [has patch][needs review[necko-backlog]
Whiteboard: [has patch][needs review[necko-backlog] → [has patch][needs review][necko-backlog]
Patrick, this bug has an 18 month-old patch pending. Do we take the patch after updating it, or is this a WONTFIX situation?
Flags: needinfo?(mcmanus)
Priority: P1 → P3
The comments are marked as "spam" because they do not belong in a bug tracker and are not helping in any way the resolution of the bug, other than criticizing (just like you are doing).
If you think this feature is really important, commenting on Bugzilla is not the right way to get your voice heard.
It's not spam, but it's definitely an abusive comment and an unacceptable use of Bugzilla. I encourage you to review Bugzilla's Etiquette and Contributor Guidelines, available here:

https://bugzilla.mozilla.org/page.cgi?id=etiquette.html

... and I urge you to take their advice to heart if you'd like to continue participating in Bugzilla.

I have restricted this bug to editbugs-privileged users only. If anyone has something they would like to add to this bug that would materially move it towards a resolution or has any futher concerns please fell free to contact me directly. 

Thank you.
Restrict Comments: true
After asking around, we don't really have a compelling usage or feature that depends on this in Firefox and don't foresee that usecase emerging, particularly in light of our move to the WebExtensions API.

I'm shuttering this bug; sorry folks, but if we find a compelling reason to revisit this we should probably have that discussion in a new bug with a well-defined idea of what success looks like.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
A lot of IETF standards depend on this.

Success is:
* Supporting service discover
* Being able to implement XMPP (correct implementation is currently impossible - that's why I developed the patch here)
* Being able to implement service discovery like Exchange AutoDiscover
* Thunderbird account creation dialog needs MX server queries. We currently use a hack to go around this limitation in core.

This bug is a requirement to implement a lot of Internet standards.

If WebExtensions can't do this, then they cannot implement many of the Internet standards, e.g. cannot implement a standards-based chat extensions. So, they need a corresponding API. For the API to exist, Mozilla first needs the underlying implementation. This bug adds it.

This bug has a patch that I developed and that took me a lot of time to develop.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Some IETF standards that depend on this:
1. NAT Traversal (RFC 8445, RFC 7350, etc.)
2. SIP service location (RFC 3263)
3. Timezone data distribution (RFC 7808)
4. XMPP (RFC 7712, RFC 7590, etc)
5. email protocol service location (RFC 6186)

About 200 IETF RFCs contain the string " SRV ". It should be possible to implement these standards in a browser without a kludge (requiring a different way of publishing such data) to workaround the missing functionality.
While all of that is true, none of it speaks to any feature that is currently planned to be part of Firefox, nor are they supported in any other browser. If that changes, we can definitely reconsider.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → WONTFIX
> none ... currently planned to be part of Firefox

Your decision also implies that a WebExtension *cannot* properly implement XMPP (chat), or any of the other protocols.

Situation now is much better than when I wrote the patch here. Mozilla current already supports TXT, so I don't see why SRV should not be allowed. It should be a trivial addition now.
Assignee: nobody → ben.bucksch
You need to log in before you can comment on or make changes to this bug.