1313476 - VPN access to kms1.ad.mozilla.com for taskcluster-worker (packet.net and dev-setup)

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: NetOps: DC ACL Request, task)

Description

[Jonas Finnemann Jensen (:jonasfj)]

I want to experiment with windows VMs managed by taskcluster-worker on packet.net.

For this I've learned that my VMs will need access to:
  kms1.ad.mozilla.com:1688

Since I fully control the host at hypervisor level, I should be able to insert appropriate DNS records and forward direct requests for the IP hosting kms1.ad.mozilla.com to a VPN connection.

So I need:
 1) A VPN user with access to kms1.ad.mozilla.com:1688
 2) Credentials/keys for said VPN user

My plan is to store the credentials in the taskcluster-secrets service.
Load them into my worker host, have it setup and VPN, DNS and iptables,
such that VMs can activate.

I'll probably use the same setup developing locally. If this is sensitive access
two VPN clients one for development and one for production would be ideal.
Just to make rotation of development keys easier.

For now I would be happy with one.
Any questions I'm jojensen on LDAP, jonasfj on IRC, please ping me.

Comment 1

[Amy Rich [:arr] [:arich]]

Ccing Q since he runs the KMS server.

Comment 2

[Dave Curado :dcurado]

Hello Jonas,
Can you tell me more about the connectivity you require?

Let's start with your VMs... where are they physically located? 

Thanks,
Dave

Comment 3

[Jonas Finnemann Jensen (:jonasfj)]

Machines are physically located in packet.net
For development some will be located on my laptop, wherever I am.

I would really prefer not relying on IP address ranges as that makes it hard to deploy
with a new provider, and causes unpredictable issues when providers update IP ranges.

I run everything in a docker container, both locally and in packet, so I'll probably do a VPN
setup in my container. That means that I'll have one VPN connection per worker.

Note: the task code I run will be running inside VMs, so they won't be able to steal VPN keys.

Comment 4

[Jonas Finnemann Jensen (:jonasfj)]

Just to clarify:

I buy a bare metal machine, which runs:
 - taskcluster-worker, which does:
   - iptables setup
   - runs custom DNS server for VMs
   - runs a QEMU VM per task 
So I'm in full control of the bare-metal machine on which the VMs are running :)

Comment 5

[Dave Curado :dcurado]

For you VPN connection, are you thinking openvpn?
I think that is what would make the most sense -- it's the VPN solution we have in place.

If you can do that, I think this will be pretty easy.
I'm sitting in my hotel room, connected to openvpn, and...

/Users/dcurado. host kms1.ad.mozilla.com
kms1.ad.mozilla.com has address 10.22.69.24

/Users/dcurado. ping kms1.ad.mozilla.com
PING kms1.ad.mozilla.com (10.22.69.24): 56 data bytes
64 bytes from 10.22.69.24: icmp_seq=0 ttl=126 time=49.334 ms
^C

/Users/dcurado. telnet kms1.ad.mozilla.com 1688
Trying 10.22.69.24...
Connected to kms1.ad.mozilla.com.
Escape character is '^]'.

Please let me know what you think?
Thanks.

Comment 6

[Jonas Finnemann Jensen (:jonasfj)]

dcurado:

openvpn sounds perfectly awesome! :)

If you can set me up with keys/config-options for openvpn then I'm sure I'll figure out the rest.

Comment 7

[Dave Curado :dcurado]

Jabba -- would we be able to provide an openvpn connection for Jonas?
Thank you.

Comment 8

[Justin Dow [:jabba]]

I believe we can make this work. The difficulty here is normally the MFA part of openvpn, which is difficult to automate, however there is a precedent. There's a user called "taskcluster-balrog" that I believe does something similar (automated headless vpn connection), and we should be able to make a similar user for this use case. Should I call this account taskcluster-worker? Or some other descriptive name?

Comment 9

[Dave Curado :dcurado]

Works for me.
Thank you.

Can you send whatever login/auth info to Jonas once you've put it together?
Thank you!

Comment 10

[Justin Dow [:jabba]]

Yep. Jonas, can you confirm the name of the user account? Then I'll get started creating the account and certs and set up the access rules, etc.

Comment 11

[Jonas Finnemann Jensen (:jonasfj)]

Awesome, I think a good name would be:

  taskcluster-worker-kms

As this is specific to KMS access.

Comment 12

[Justin Dow [:jabba]]

I got the ldap account and vpn config set up and sent to :jonasfj (gpg encrypted) and the setup is pretty identical to the taskcluster-balrog setup, although I created a new vpn group for this one called vpn_taskcluster_kms, which solely grants access to the host/port listed in comment 0. It is necessarily also a member of vpn_default, which should allow it to query our internal DNS servers in order to resolve the hostname. I think this is good to go. :jonasfj will play with it and let me know if things don't work as expected.