Update search datasets to reflect with_ads/ad_click scalar changes in Firefox 86
Categories
(Data Platform and Tools :: General, defect, P1)
Tracking
(Not tracked)
People
(Reporter: standard8, Assigned: benwu)
References
Details
In Firefox 86 (bug 1664847) we have landed a change to which separates the browser.search.with_ads
and browser.search.ad_clicks
into multiple scalars, where the individual scalars reference the search access point (SAP).
For a breakdown of the new scalars please see here: https://firefox-source-docs.mozilla.org/browser/search/telemetry.html#searchserptelemetry-jsm
The sum of the new scalars should be equal to the old scalars. Due to this, the intent is to remove the old scalars around Firefox 88 as that is effectively duplicate data.
We will need search_clients_daily
and search_aggregates
updated as appropriate to handle these changes. We may want to backfill to when the additions landed.
Updated•4 years ago
|
Hi Mark, would it be possible to hold off on removing the scalars for a bit? A number of reasons:
- This is my last week at Mozilla and Revenue Data Engineering is a bit under-resourced at the moment as a result
- There's a bit more discussion that needs to happen, I think, regarding how to ensure continuity in the data pipeline since we still need to handle pings from users that are on versions before 86
- Due to some historical baggage that we're working on deprecating, we need to make changes to the underlying dataset that powers
search_clients_daily
before making the actual changes to migrate, so this is a bit more work than might seem at first glance
Reporter | ||
Comment 2•4 years ago
|
||
I would be fine for that, from the code perspective this is really just a couple of extra lines of code. Obviously there's effectively duplicate telemetry being logged & handled though.
I think if we could add dependencies to/where to watch for the work (so that we have somewhere to track) and this is added to the schedule for after they are done then that should be fine.
Reporter | ||
Comment 3•4 years ago
|
||
Just to note that bug 1688599 is the bug filed to remove this, I'm adding this as blocking that one.
Reporter | ||
Comment 4•4 years ago
|
||
Bug 1688599 has just added some additional keyed scalars (in Firefox 89) on top of the ones added in bug 1664847. The docs should be updated in the next day or two.
FTR, the new scalars added in bug 1688599 are:
- browser.search.withads.tabhistory
- browser.search.withads.reload
- browser.search.adclicks.tabhistory
- browser.search.adclicks.reload
Assignee | ||
Comment 5•3 years ago
|
||
This was done with https://github.com/mozilla/bigquery-etl/pull/2253
See https://mozilla-hub.atlassian.net/browse/DS-1603 for some more context
Updated•2 years ago
|
Description
•