1398873 - [meta] Establish a probe info service

Status: RESOLVED FIXED

(Data Platform and Tools :: General, enhancement, P3)

Description

[Georg Fritzsche [:gfritzsche]]

We repeatedly come up with the need for a service or dataset that provides us with:
- a view of all probes (histograms, scalars, events) of a Firefox release
- a historical view of the above, covering at least major versions (future goal: build ids / revisions)
- a single place to store this and hide the implementation details of scraping & parsing Histograms.json/Scalars.yaml/...

Comment 1

[Georg Fritzsche [:gfritzsche]]

probe-scraper delivers a first part of that, pulling the scraping and parsing together in one place:
https://github.com/mozilla/probe-scraper

Comment 2

[Georg Fritzsche [:gfritzsche]]

There is also potential overlap with buildhub:
https://github.com/mozilla-services/buildhub/

Could this data fit in there?

Comment 3

[Mark Reid [:mreid]]

The prototype from comment 1 above has been extended quite a bit and is already very useful:

https://georgf.github.io/fx-data-explorer/index.html

Georg, do you think we can close this bug?

Comment 4

[Georg Fritzsche [:gfritzsche]]

I don't think we have a reliable solution in place yet for the pipeline to use.
I scheduled a meeting this week to talk through what this will take and will follow up from there.

Comment 5

[Mark Reid [:mreid]]

We now have a reliable solution, used internally by the mozilla-schema-generator among others.