Add annotations for non-public histograms/scalars/events etc

RESOLVED WONTFIX

Status

()

Toolkit
Telemetry
RESOLVED WONTFIX
a year ago
a year ago

People

(Reporter: mreid, Unassigned)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(firefox53 affected)

Details

User Story

Some scalars and histograms need to be excluded from the aggregates dataset. We need this initially for search scalars, which should not be public facing.
(Reporter)

Description

a year ago
It would be nice to be able to add something to the definitions for histograms and scalars (and future data types) to indicate whether or not they should be included on the public facing aggregations used on telemetry.mozilla.org.
I wonder where the best location for that is.

The client definitions for measurements (e.g. Histograms.json) don't seem like the right place:
For the non-public information, we want to hide all the data we ever see.
However, for the definitions we usually want to look at the client definition for the revision of the Firefox build.

I can see this living in a separate file in the client source, with the pipeline (aggregator et al) pulling it from mozilla-central tip.
But maybe it's easier to just have that configuration live in a pipeline repo or so?
Flags: needinfo?(mreid)
Flags: needinfo?(fbertsch)
User Story: (updated)
Flags: needinfo?(fbertsch)
(In reply to Georg Fritzsche [:gfritzsche] from comment #1)
> I wonder where the best location for that is.
> 
> The client definitions for measurements (e.g. Histograms.json) don't seem
> like the right place:
> For the non-public information, we want to hide all the data we ever see.
> However, for the definitions we usually want to look at the client
> definition for the revision of the Firefox build.
> 
> I can see this living in a separate file in the client source, with the
> pipeline (aggregator et al) pulling it from mozilla-central tip.
> But maybe it's easier to just have that configuration live in a pipeline
> repo or so?

I don't know if I like having yet *another* place with information about measurements. If it were me, it would be Histograms.json (and the yml file for scalars).
(Reporter)

Comment 3

a year ago
I was initially thinking that it *should* live with the client definitions, just so that when they were defined and implemented, the decision re: public could be made at the same time.

It could just as well live as a config file with the aggregation code, though I don't know that that would be a significant improvement over what we have now.

Maybe we don't even need a general solution to something this specific - it appears there is only one exception for histograms[1]. If there is likewise a single set of scalars to be excluded from the public aggregates, I'm fine to WONTFIX this bug.

[1] https://github.com/mozilla/python_mozaggregator/commit/f29601ff13a2fe4fe2769540a304d1bf53daebc3
(Reporter)

Updated

a year ago
Flags: needinfo?(mreid)
(In reply to Mark Reid [:mreid] from comment #3)
> I was initially thinking that it *should* live with the client definitions,
> just so that when they were defined and implemented, the decision re: public
> could be made at the same time.
> 
> It could just as well live as a config file with the aggregation code,
> though I don't know that that would be a significant improvement over what
> we have now.
> 
> Maybe we don't even need a general solution to something this specific - it
> appears there is only one exception for histograms[1]. If there is likewise
> a single set of scalars to be excluded from the public aggregates, I'm fine
> to WONTFIX this bug.
> 
> [1]
> https://github.com/mozilla/python_mozaggregator/commit/
> f29601ff13a2fe4fe2769540a304d1bf53daebc3

I'm thinking this may be the right answer. It doesn't seem we need to generalize this very much right now, and as we start to think about moving people away from TMO to other tools, it will become even less relevant.
Sounds good to me, lets revisit this when this is an actual issue.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
Resolution: FIXED → WONTFIX
You need to log in before you can comment on or make changes to this bug.