1424411 - Put a plan in place for removing activity_date from client_count_daily

Status: RESOLVED FIXED

(Data Platform and Tools Graveyard :: Datasets: Client Count, enhancement, P2)

Description

[Daniel Thorn [:relud]]

Put a plan in for removing activity_date altogether (e.g. once 99.5% of pings have equivalent activity_dates and submission_dates).

as requested by :frank in 1380640

Comment 1

[Daniel Thorn [:relud]]

as per bug 1422892 I think we should just remove activity_date from client_count_daily immediately, and let people know when they switch that they need to use `submission_date` instead of `activity_date`.

Frank, Mark, thoughts? objections? I saw :flod is using client_count_daily now in https://sql.telemetry.mozilla.org/queries/50134/source but I think that relatively few (maybe 4?) people have touched client_count_daily right now, so I think we would benefit from making this switch sooner rather than later.

Comment 2

[Daniel Thorn [:relud]]

> and let people know when they switch

switch from `client_count` to `client_count_daily`

Comment 3

[Frank Bertsch [:frank]]

(In reply to Daniel Thorn [:relud] from comment #1)
> as per bug 1422892 I think we should just remove activity_date from
> client_count_daily immediately, and let people know when they switch that
> they need to use `submission_date` instead of `activity_date`.
> 
> Frank, Mark, thoughts? objections? I saw :flod is using client_count_daily
> now in https://sql.telemetry.mozilla.org/queries/50134/source but I think
> that relatively few (maybe 4?) people have touched client_count_daily right
> now, so I think we would benefit from making this switch sooner rather than
> later.

I think this is a good idea. We'll need two things:
1. Documentation describing how submission_date based client_counts have changed over time (e.g. when ping sender came into effect) such that they are now acceptable to use. We can then just point people at that documentation as they switch over
2. A version bump, and a backfill of all the old data

Comment 4

[Mark Reid [:mreid]]

This sounds like a good idea. Let's leave the v1 data in place so we can still do comparisons between submission date and activity date on historic data, and add v2 and backfill it per above.

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/telemetry-airflow/pull/220

Comment 6

[Daniel Thorn [:relud]]

bumping this to a p2 so we can bikeshed bug 1434991 into the change

Comment 7

[Daniel Thorn [:relud]]

client_count_daily now has this change