1482924 - Investigate backfilling Savant data to correct active_ticks values

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P1)

Comment 1

[bugzilla]

The Savant subsession-split meta-event uses the active_ticks value from simpleMeasurements, which is incorrect in ~8% of ping per bug 1482466. Thankfully, since this study ran entirely on FF 61, we can use the scalar value, which is correct.

I'll leave the investigation re: the best way to go about this to Jeff but since even a full backfill would not take too long, this seems like a clear-cut case where a full mitigation makes sense.

Comment 2

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/telemetry-streaming/pull/171

Comment 3

[Jeff Klukas [:klukas] (UTC-4)]

Discussed with Sunah, Josephine, and folks on the Amplitude side.

Looks like there's not really a concept of deleting events in Amplitude, so folks on the Amplitude side were recommending we start with a fresh project and backfill everything. That would require some coordination to update names of projects, API keys, etc., and would also mean blowing through another ~1 billion events of our annual quota (35 billion).

I think I'm going to pursue a solution for a "partial" backfill of just session split events, since those are the only ones with active_ticks populated, which is the affected quantity. There are only ~15 million of these events, so will have much less effect on the quota.

The current events are called "Meta - session split" and in the backfill, I'll plan to change the name to "Meta - session split v2". Once the code is updated for the new name, I'll backfill and then we can either set the old event to be inactive and not visible, or delete the old event type or set up a "data filter" to hide it.

See https://amplitude.zendesk.com/hc/en-us/articles/235649848-Settings

Comment 4

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/telemetry-streaming/pull/172

Comment 5

[Jeff Klukas [:klukas] (UTC-4)]

Kicked off on ATMO:

spark-submit --class com.mozilla.telemetry.streaming.EventsToAmplitude telemetry-streaming.jar --config-file-path savant2.json --url https://api.amplitude.com/httpapi --from 20180626 --to 20180814 --max-parallel-requests 40

Where savant2.json has all events stripped out except for the session split event.

Savant_Prod is showing 704,698,214 right now, and I expect the to increase to ~720M once this completes.

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/telemetry-streaming/pull/173

Comment 7

[Jeff Klukas [:klukas] (UTC-4)]

Ran into issues with deduplication, so had to make another logic change. Trying another backfill in increments.

Comment 8

[Jeff Klukas [:klukas] (UTC-4)]

The v3 event backfill looks good. I've changed names in the UI so that the v1 and v2 events are hidden and the v3 event shows up as "Meta - session split".