1387163 - Estimate effort needed to report search volume in experiments

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, enhancement, P1)

Description

[Ryan Harter [:harter]]

I'm interested in testing whether a Firefox feature change affects search volume (roughly: the sum of all SEARCH_COUNTS). Is this something that the experiment viewer can assist with?

I took a quick look at the telemetry-batch-view job and it looks like it's dealing primarily with histograms. Does it work well with keyed histograms? 

To be clear, I'm not asking you to implement this. I recognize this will take an additional allocation of resources to implement.

Comment 1

[Frank Bertsch [:frank]]

Yes! Currently keyed histograms are "unkeyed" so to speak, where all key histograms are aggregated together (so {a: {1:1}, b: {2:2}} would just become {1:1, 2:2}). All we have to do is add the "SEARCH_COUNTS" histogram to the histogram whitelist, and it should show up during the next run.

The viewer will show the two histogram distributions. If you need more details on what exactly is being shown let us know.

Comment 2

[Ryan Harter [:harter]]

Gah, I realize now that this won't work. SEARCH_COUNTS contains keys for (e.g.) hooli.incontent and hooli.sap which count mostly the same thing. Our metrics would be uninterpretable without some filter on which keys are included.

Comment 3

[bugzilla]

We've discussed exploding out keys as separate aggregates in the past -- do you have a timeline for when you want to launch this?

Comment 4

[Ryan Harter [:harter]]

I'm trying to scope out a goal for Q3.

Comment 6

[bugzilla]

:harter -- is this bug still valid?

Comment 7

[Ryan Harter [:harter]]

Nope.