1415935 - Replace Client Count dataset with incremental version

Status: RESOLVED FIXED

(Data Platform and Tools Graveyard :: Datasets: Client Count, enhancement, P1)

Description

[Mark Reid [:mreid]]

A new HLL-based client count dataset was introduced in Bug 1412971.

Let's validate its correctness and performance, make sure it's backfilled as far back as makes sense, and finally deprecate the current client_count dataset in favour of the new one.

Comment 1

[Mark Reid [:mreid]]

The client_count_daily dataset is backfilled to Nov 2015, which is as far back as we can reasonably go.

Comment 2

[Ryan Harter [:harter]]

Can we prioritize this work over the next sprint? I'm seeing a lot of chatter on clients_daily and the data looks misleading or malformed. For example, it's unlikely that this query reflects reality [1]. 

We should at least mark clients_daily as deprecated in docs.tmo. We should probably redirect folks to clients_count_daily now since it's providing more reasonable results.

[1] https://sql.telemetry.mozilla.org/queries/52399/source#139629

Comment 3

[Daniel Thorn [:relud]]

clients_daily is a different data set, which is being updated in bug 1426172

that said, this can be worked on next sprint, now that the main_summary backfill is being copied over tonight.

ni? mreid - can this and related bugs be a p1 for next sprint?

Comment 4

[Ryan Harter [:harter]]

Sorry, I meant to say client_count.

Comment 5

[Mark Reid [:mreid]]

Yes, let's switch things over to the incremental dataset.

Comment 6

[Daniel Thorn [:relud]]

The client count dashboard has been updated, client_count_daily has been backfilled, docs.tmo says client_count is deprecated[1], and [e-mail] has been notified of the deprecation.

[1] https://github.com/mozilla/firefox-data-docs/pull/114

I don't think bug 1394547 actually blocks closing this.