Telemetry Server cutover checklist

RESOLVED FIXED

Status

Webtools
Telemetry Server
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: mreid, Assigned: mreid)

Tracking

Trunk
x86_64
Linux
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

4 years ago
Oct 1, 2013 - Telemetry Server cutover plan
===========================================

1.  Set up production EC2 telemetry-server instances and Load Balancer
2.  Run tests on servers
3.  Clean prod S3 buckets / SQS Queue
4.  Cut over DNS for incoming.telemetry.mozilla.org to point to production AWS LB
5.  Verify we're getting data
6.  Remove "cturra" header for the Zeus rule (and update target) to redirect from data.mozilla.com/submit/telemetry to incoming.telemetry.mozilla.org (see Bug 916243)
7.  Verify no more data is going to Bagheera / Kafka
8.  Shut down the relay infrastructure
9.  Verify everything is working as expected
10. Switch http://telemetry-dash.mozilla.org/ to point at S3
11. Clean up

Bugs with details on the above points to follow.
(Assignee)

Updated

4 years ago
Assignee: nobody → mreid
Blocks: 911300
(Assignee)

Updated

4 years ago
Depends on: 921164
(Assignee)

Updated

4 years ago
Depends on: 921167
(Assignee)

Updated

4 years ago
Depends on: 921170
(Assignee)

Comment 1

4 years ago
(In reply to Mark Reid [:mreid] from comment #0)
> 1.  Set up production EC2 telemetry-server instances and Load Balancer
> 2.  Run tests on servers
> 3.  Clean prod S3 buckets / SQS Queue
These three points are covered by bug 921164

> 4.  Cut over DNS for incoming.telemetry.mozilla.org to point to production
> AWS LB
> 5.  Verify we're getting data
These two follow on work in bug 911309, the actual change to be done is bug 921167

> 6.  Remove "cturra" header for the Zeus rule (and update target) to redirect
> from data.mozilla.com/submit/telemetry to incoming.telemetry.mozilla.org
> (see Bug 916243)
As stated, this is bug 916243

> 7.  Verify no more data is going to Bagheera / Kafka
> 8.  Shut down the relay infrastructure
> 9.  Verify everything is working as expected
These three are bug 921170

> 10. Switch http://telemetry-dash.mozilla.org/ to point at S3
This is bug 921077

> 11. Clean up
I'll take care of cleanup as part of closing this bug, and it will include
things like:
- Shut down prototype "relay" server instance (sending to mreid-telemery-incoming)
- Migrate S3 data from mreid-telemetry-published to production telemetry bucket
(Assignee)

Comment 2

4 years ago
> 10. Switch http://telemetry-dash.mozilla.org/ to point at S3

This doesn't need to block the "cutover" state, so it can be handled separately.

> 11. Clean up
Relay instances have been shut down. Data doesn't need to be migrated from the pre-release bucket, since we will be exporting data from Mango (per Bug 922745) and don't want to end up with duplicates from when the systems were running in parallel.

I'm calling this bug "done". Thanks to everyone who helped!
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.