New VM for Firefox Health Report: Kafka -> AWS Relay

RESOLVED FIXED

Status

Infrastructure & Operations
Virtualization
RESOLVED FIXED
4 years ago
10 months ago

People

(Reporter: mreid, Assigned: cknowles)

Tracking

Details

(Whiteboard: [vm-create:1])

(Reporter)

Description

4 years ago
I would like to provision a VM which reads data from Bagheera's Kafka queue and sends it to Amazon S3 for processing.  This is part of bug 1043504.

The machine would need access to 
node*.bagheera.metrics.scl3.mozilla.com:2181
node*.bagheera.metrics.scl3.mozilla.com:9092

And outbound access to AWS (either direct or using proxy.dmz.scl3.mozilla.com).

Please create the VM outside the metrics vlan and generate ACLs to connect to the bagheera nodes.

Desired server specs:
Hostname: fhrrelay
Memory: min 4GB
Disk: min 30GB
CPU: 4 vcpus, fairly beefy since they'll be doing a lot of compression
OS: Ubuntu server

Fast networking is also desirable, but I'm not sure if that's tunable for a VM.

I'd like to be able to connect to this machine using ssh.mozilla.com as a jumphost.
(Assignee)

Comment 1

4 years ago
per IRC conversation with :tmary - he'll be filing the bugs for the netflows (main reason virt usually doesn't is that we're not the service owners - should netops/opsec have questions, we're not the right ones to answer them)

Did have one other question - who needs access to this VM?  I assume mreid needs root, does anyone else?  (:tmary marked himself as optional)

I'm hoping to have the initial stubbed in VM ready for further configuration (flows/access/etc) by tomorrow at the latest.

Here's the name/specs I'm shooting for:
Hostname: fhrrelay1.private.scl3.mozilla.com
Memory: 4GB
Disk: Default (40GB)
CPU: 4 vcpus 
OS: Ubuntu server

Let me know if there are concerns with this.
Assignee: server-ops-virtualization → cknowles
(Assignee)

Comment 2

4 years ago
Alright, the VM "fhrrelay1.private.scl3.mozilla.com" exists at IP "10.22.75.192" - which should be enough to start moving on the netflows.

As to the access - let me know the list of who needs what level (sudoers/mortal-user) of access so I can file that bug.
(Reporter)

Comment 3

4 years ago
Thanks, this sounds good to me.  Please give tmary sudoer access too.

Updated

4 years ago
Depends on: 1058569
(Assignee)

Updated

4 years ago
Depends on: 1058871
(Assignee)

Comment 4

4 years ago
I think I saw that :mreid now has access - assuming that bug 1058871 gets some love soon, :tmary should also get access.
(Assignee)

Comment 5

4 years ago
And now tmary does as well - anything else you need from us?
(Reporter)

Comment 6

4 years ago
Looks good, thanks!
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Updated

4 years ago
Whiteboard: [vm-create:1]
Product: mozilla.org → Infrastructure & Operations

Updated

10 months ago
See Also: → bug 1360646
You need to log in before you can comment on or make changes to this bug.