1349995 - change hostname in logs to something more helpful [antenna]

Status: RESOLVED FIXED

(Socorro :: Antenna, task)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

The logging infrastructure in Antenna uses socket.get_hostname() which apparently returns a hostname that's like the internal ip of the node. That's not helpful since it's not what Datadog uses which means we have to figure out which node it is in the EC2 listing and if the node is gone (scale down, deploy, etc), then we are out of luck.

This issue covers figuring out how to get the host that Datadog uses in the logs.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Miles would prefer a public dns moniker since it makes it easier for him to ssh into them.

Boto has a way of getting instance metadata. Miles suggested something like this:

> >>> macs = boto.utils.get_instance_metadata()['network']['interfaces']['macs']
> >>> for mac in macs:
> ...   print macs[mac]['public-hostname']
> ...
> ec2-34-199-122-62.compute-1.amazonaws.com

but we're using boto3 and it doesn't have that. boto/boto3#313 . Might be easy to reimplement it--I'd have to look into it.

We want to make sure whatever we do works in AWS but also on the local host.

We could put the things in the environment and Antenna could pick them up at startup.

This is worth figuring out because we'll use the same thing for the processor.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Implementing get_instance_metadata doesn't look super hard. Looks like it's just doing an API call.

https://github.com/boto/boto/blob/develop/boto/utils.py#L379

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Miles said he'd put something in the environment to get. As soon as I find out what that variable is, I'll make the changes.

Comment 4

[Miles Crabill [:miles]]

The env vars are PUBLIC_HOSTNAME and INSTANCE_ID, corresponding to the same values in EC2.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

Instead of having two environment variables, can we have one variable that has whatever we want in it? So instead of:

    PUBLIC_HOSTNAME='jim'
    INSTANCE_ID='bob'

we'd have:

    HOST_ID='jim bob'

I think that's a lot easier to think about in the code and documentation. Plus it makes it easier on the ops side if we ever want to adjust the HOST_ID naming scheme.

Miles: Can we make this change?

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: Link to Github pull-request: https://github.com/mozilla/antenna/pull/191

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

This landed. Miles set HOST_ID in the environment and I see it in the logs. So we're all set here.

Marking as FIXED.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Switching Antenna bugs to Antenna component.