Remove Pub/Sub sink for FxA logs related to AET
Categories
(Cloud Services :: Server: Firefox Accounts, task)
Tracking
(Not tracked)
People
(Reporter: klukas, Assigned: jbuck)
References
Details
From :whd in https://bugzilla.mozilla.org/show_bug.cgi?id=1697602#c5
For the ops side bug we're going to need to coordinate with FxA-side efforts WRT the pubsub topic. It may be that we decide to keep the topic in low retention (10m) mode as well as the edge config as the way to discard AET messages, but we certainly don't want to remove the topic until the FxA-side log sink is decommissioned
We'd like jbuck to remove the log sink to Pub/Sub that was used for feeding FxA-side AET messages to the data pipeline, so that we can delete the topic and the Dataflow jobs associated with the AET path.
:whd should correct me if I'm wrong, but I don't think there are any blockers for moving ahead with this. jbuck can remove the sink at will, and we'll see the publish rate to the topic go to zero, at which point we'll be able to proceed with data pipeline-side decomm steps.
|
||
Comment 1•4 years ago
|
||
There are no blockers to removing the sink (this bug), but we will need to decide how to deal with AET data from old clients before removing the edge routing and topic (not this bug). My guess is that we'd implement a MessageShouldBeDroppedException for AET payloads (utilized by both the telemetry and structured decoders) but a low-tech solution is to leave the pubsub topic around with very low retention.
|
Assignee | |
Comment 2•4 years ago
|
||
I've removed both stage and prod log sinks
|
|
Assignee | |
Updated•4 years ago
|
Description
•