544274 - fix reverse DNS entries on talos r3 machines

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations: Projects, task)

Description

[[e-mail] (:bhearsum)]

These machines all still have the hostname of the ref machine, which means they're not connected to the Puppet server properly. They either need to get their hostnames set by DNS properly or we need to go around and change them manually.

They should be:
talos-r3-fed-NNN.build.mozilla.org
talos-r3-fed64-NNN.build.mozilla.org

We also need to make sure that future deployments have their hostname set from the git go.

Comment 1

[[e-mail] (:bhearsum)]

Phong and I looked into this a bit and first of all, discovered that HOSTNAME was forcibly set in /etc/sysconfig/network. Getting rid of that didn't fix anything, though. The current theory is that there might be reverse entries in both mv and build DNS:
14:33 < phong> it could be that we have both in MV dns and build.mozilla.org  
               maybe i need to remove them from the mv domain.

Comment 2

[Phong Tran [:phong]]

I just talked to dmoore about this.  It's a problem with reverse dns for the hosts living in Mountain View and they need in the build.mozilla.org domain.  In the mean time, the fastest way to fix this is to manually set the hostnames in /etc/sysconfig/network.  Or you can wait until we get this fixed.

Comment 3

[[e-mail] (:bhearsum)]

(In reply to comment #2)
> I just talked to dmoore about this.  It's a problem with reverse dns for the
> hosts living in Mountain View and they need in the build.mozilla.org domain. 
> In the mean time, the fastest way to fix this is to manually set the hostnames
> in /etc/sysconfig/network.  Or you can wait until we get this fixed.

I updated these machines manually as a quick fix. We need to make sure this step gets into the standard cloning procedures until the root problem is fixed, though.

Comment 4

[Derek Moore [:dmoore]]

root cause: The MPT DNS server (which is used by all build clients) doesn't know how to perform reverse DNS lookups for 10.250.48.0/23. 

MV knows, MPT doesn't. We're pondering how to fix this.

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #4)
> root cause: The MPT DNS server (which is used by all build clients) doesn't
> know how to perform reverse DNS lookups for 10.250.48.0/23. 
> 
> MV knows, MPT doesn't. We're pondering how to fix this.

Any update on this? We've another batch of minis coming online, we'll hit this again soon.

Comment 6

[Dave Miller [:justdave]]

colo dns config repo now pulls in the office zones as an svn:external and the config file for MPT now loads the reverse zones from it.  Forward lookups don't work yet, that'll need some massaging on the office zones to prevent conflicts first.

X       offices
M       config/view.mpt-dmz
A       config/zones.mvoffice-reverse
Sending        dnsconfig/config/view.mpt-dmz
Adding         dnsconfig/config/zones.mvoffice-reverse
Transmitting file data ..
Committed revision 5932.

Comment 7

[Arzhel Younsi [:XioNoX]]

This bug is about reverse DNS, so I thing we can close it.

Please file a new bug or reopen if you need forward lookups.