Closed Bug 807494 Opened 10 years ago Closed 9 years ago
Set up public stun servers for Web
WebRTC NAT traversal requires a STUN (or turn) server so clients can resolve their public IP address. This bug is about setting up a distributed network of such servers for our users.
I plan to set up some servers for testing on Amazon EC2, with traffic distributed by geo-variant dns responses. The servers would be public for anyone to use, but the primary intent is to have something we can test against as we develop and deploy WebRTC where we can control the availablity and privacy protections.
Basic plan is to set up an instance running the default 'stun-server' package, see how that goes.
dev-media post for wider discussion: http://thread.gmane.org/gmane.comp.mozilla.devel.media/141 or https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.media/1p4eLDOOM9g
I tried using boxgrinder.org to generate Scientific Linux VM images. Code for this is at https://github.com/mozilla/stun-vm Unfortunately, SL 6.3 doesn't include a stun server package, which removes most of the impetus for using a long-support-lifetime distribution. There are also so drawbacks with boxgrinder: - it has to run as root - docs suggest there can be information leakage between the host environment and the image - it makes images with a default root password and doesn't set 'PermitRootLogin no' in sshd_config So it looks like this isn't a great way to go.
Next I tried installing the 'stun' package on an official ubuntu 12.04 ec2 image. I immediately hit the snag that stund wants two independent IP addresses so it can determine if the client is behind symmetric NAT, and ec2 only provides a single address by default. This can be worked around with the new "Virtual Private Cloud" features, which allow assigning multiple private IP addresses, and then mapping public IP addresses to them. More complicated to set up; I don't have automation written for this yet. With this procedure I got the instance to respond to stun queries: $ stun-client 18.104.22.168 STUN client version 0.96 Primary: Indepndent Mapping, Port Dependent Filter, random port, no hairpin Return value is 0x000016 Next issue is with ec2 instances always being behind NAT. We'll need to hack the server to know what public addresses the internal ones map and use those instead when constructing response packets, to remove the EC2 NAT from the measurement.
One other issue that juberti just brought up at the mmusic meeting is that AWS only has one datacenter in Asia, and it's in Singapore, which means that anyone trying to set up a call from India is going to have 500 ms RTTs.
That is an excellent point. (There's also one in Tokyo, but that's likely even further away from India.) Do you have an alternative suggestion? I am wary of looking for an established service partner for our default, because of the privacy implication for our users. Longer term we can of course find dedicated server space in more locations, but I think AWS is still helpful to get something up quickly.
AWS also doesn't support ipv6.
To clarify: STUN needs two public IP addresses. This is possible with AWS by launching an instance on a private subnet and connecting two elastic IPs. This is what I've done for the one available at 22.214.171.124. Abr has confirmed this setup works with our current code. The stun-server software itself needs patching, to handle this cleanly, I'll open a new bug for that.
This patch isn't intended to be checked in. I'm attaching it as a convenience for anyone who wants to move over to the new server before Bug 825703 lands.
Note you won't be able to use that IP above in #c9 - we'll have to bring up this instance in a Service VPC, where we can't take that IP. Can we use a FQDN in there instead of an IP?
Not right now, no, because we don't support FQDNs for STUN servers.
Mark: I think we all agree that a FQDN would be preferable to an IP address. Right now, we're working to get DNS resolution into the STUN handling, but we really don't want to block getting a new STUN server into the code on finishing that task.
Comment on attachment 701941 [details] [diff] [review] Temporary patch to use new server Review of attachment 701941 [details] [diff] [review]: ----------------------------------------------------------------- adam, we should land this patch.
Attachment #701941 - Flags: review+
(In reply to Ralph Giles (:rillian) from comment #9) > To clarify: > > STUN needs two public IP addresses. This is only for NAT characterization via STUN. ICE does not rely on this. > This is possible with AWS by launching > an instance on a private subnet and connecting two elastic IPs. This is what > I've done for the one available at 126.96.36.199. Abr has confirmed this > setup works with our current code. > > The stun-server software itself needs patching, to handle this cleanly, I'll > open a new bug for that. I don't understand what the patch is here. We seem to be getting reasonable responses with the existing code.
Also, per bug 830476, we (:whd, really) have server patches we're trying in svcops to handle the AWS NAT issue. We'll work these into our automated deployment of the server. We'll update this ticket when we have a services-managed stun server up, and provide the new IP. ETA: today.
(In reply to Eric Rescorla (:ekr) from comment #15) > > STUN needs two public IP addresses. > > This is only for NAT characterization via STUN. ICE does not rely on this. That's great news, and simplifies the vm setup. I was just following the stun server docs. > I don't understand what the patch is here. We seem to be getting reasonable > responses with the existing code. I'm referring to the 10.0.0.27 (and 10.0.0.103, since this server has two public addresses) in the stun response: Received stun message: 88 bytes MappedAddress = 188.8.131.52:46281 SourceAddress = 10.0.0.27:3478 ChangedAddress = 10.0.0.103:3479 XorMappedAddress = 184.108.40.206:46281 ServerName = Vovida.org 0.96 Received message of type 257 id=1
I have set up a patched stun server running at: ec2-23-22-244-173.compute-1.amazonaws.com ip: 220.127.116.11 This is without a VPC, as per comment 15. Relevant output: $ stun-client -v ec2-23-22-244-173.compute-1.amazonaws.com 1 STUN client version 0.96 running test number 1 Opened port 19578 with fd 3 Encoding stun message: Encoding ChangeRequest: 0 About to send msg of len 28 to 18.104.22.168:3478 Got a response Received stun message: 88 bytes MappedAddress = 22.214.171.124:62379 SourceAddress = 126.96.36.199:3478 ChangedAddress = 0.0.0.0:3479 XorMappedAddress = 188.8.131.52:62379 ServerName = Vovida.org 0.96 ok=1 id=1:32:65:100:144:194:220:120:134:103:63:11:74:141:19:27 mappedAddr=184.108.40.206:62379 changedAddr=0.0.0.0:3479 Return value is 0x000000
I'm not going to be able to get online for another 3 hours or so. If someone can rev the patch to use the new IP and land it, we can get STUN working on Nightly at the next build. I'll check back later and move the ball forward if no one else has by then.
Wrong IP above. :whd fixing right now. ;)
ec2-23-21-150-121.compute-1.amazonaws.com ip: 220.127.116.11
Jan-Ivar: thanks for grabbing this and running with it! I've verified the patch using the signaling unit tests, and everything appears to be working smoothly.
Whiteboard: [WebRTC] [blocking-webrtc-] → [WebRTC] [blocking-webrtc-][leave-open]
I've taken down the experimental server at 18.104.22.168.
landed on m-c but wasn't marked resolved
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Attachment #702101 - Flags: checkin?(rjesup) → checkin+
Whiteboard: [WebRTC] [blocking-webrtc-][leave-open] → [WebRTC] [blocking-webrtc-][leave-open][qa-]
You need to log in before you can comment on or make changes to this bug.