1579266 - update telemetry socorro import to pull from GCS instead of S3

Status: NEW

(Socorro :: General, task, P3)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

In https://github.com/mozilla/telemetry-airflow/pull/594, they switched the socorro import to put the data in BigQuery. So it's pulling from the S3 bucket that we put crash data destined for Telemetry in and putting it in BigQuery.

When Socorro switches to GCP, we need to update the processor to save crash data destined for Telemetry to save it in GCS and then help the Telemetry folks to update the socorro import to pull from the right place.

Comment 1

[John Whitlock [:jwhitlock]]

We're no longer moving Socorro to GCP, but it may still make sense to put Telemetry-destined crash data in GCS. If we do that, we'll still need a GCS emulator for local development.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Migrating from AWS to GCP keeps getting kicked down the road. We do eventually want to do this, but we won't until Socorro has migrated to GCP or some compelling reason pops up.

Comment 3

[Daniel Thorn [:relud]]

Attachment: [mozilla-services/antenna] bug-1579266: support storing crashes in gcs (#977)