Closed Bug 1693149 Opened 3 years ago Closed 2 years ago

Update search datasets to reflect with_ads/ad_click scalar changes in Firefox 86


(Data Platform and Tools :: General, defect, P1)



(Not tracked)



(Reporter: standard8, Assigned: benwu)



In Firefox 86 (bug 1664847) we have landed a change to which separates the and into multiple scalars, where the individual scalars reference the search access point (SAP).

For a breakdown of the new scalars please see here:

The sum of the new scalars should be equal to the old scalars. Due to this, the intent is to remove the old scalars around Firefox 88 as that is effectively duplicate data.

We will need search_clients_daily and search_aggregates updated as appropriate to handle these changes. We may want to backfill to when the additions landed.

Assignee: nobody → bewu
Points: --- → 2
Priority: -- → P1

Hi Mark, would it be possible to hold off on removing the scalars for a bit? A number of reasons:

  • This is my last week at Mozilla and Revenue Data Engineering is a bit under-resourced at the moment as a result
  • There's a bit more discussion that needs to happen, I think, regarding how to ensure continuity in the data pipeline since we still need to handle pings from users that are on versions before 86
  • Due to some historical baggage that we're working on deprecating, we need to make changes to the underlying dataset that powers search_clients_daily before making the actual changes to migrate, so this is a bit more work than might seem at first glance
Flags: needinfo?(standard8)

I would be fine for that, from the code perspective this is really just a couple of extra lines of code. Obviously there's effectively duplicate telemetry being logged & handled though.

I think if we could add dependencies to/where to watch for the work (so that we have somewhere to track) and this is added to the schedule for after they are done then that should be fine.

Flags: needinfo?(standard8)

Just to note that bug 1688599 is the bug filed to remove this, I'm adding this as blocking that one.

Blocks: 1688599

Bug 1688599 has just added some additional keyed scalars (in Firefox 89) on top of the ones added in bug 1664847. The docs should be updated in the next day or two.

FTR, the new scalars added in bug 1688599 are:

Depends on: 1634564, 1664847
See Also: → 1702178
See Also: → 1711797
Closed: 2 years ago
Resolution: --- → FIXED
Component: Datasets: General → General
You need to log in before you can comment on or make changes to this bug.