1255458 - Validate Fennec release "core" ping submissions

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect, P1)

Description

[Georg Fritzsche [:gfritzsche]]

In bug 1247605 we created a notebook to run various checks on incoming Fennec "core" pings.

We should use this as a basis to check the "core" ping data on release once we have a few days of release data from Fennec 45.

Comment 1

[Georg Fritzsche [:gfritzsche]]

I'll also re-run the ping volume analysis i did.

Comment 2

[Georg Fritzsche [:gfritzsche]]

This is a re-run of the validation from bug 1247605 with some added details:
https://gist.github.com/georgf/c9f4824e5d3bbf112db7

Notable bits:
* the count of clients seems surprisingly low (only about ~201,000 as of today)
* we have a smaller quality issue with profileDate:
  * 161 times it is missing
  * 641 times it is < 10957 (but >= 0) (i.e. a date before the year 2000)
  * 101 times it is > 17167 (i.e. a date with year 2017+)
  * Examples of those issues are printed at [54]

I'll file bugs on those.

Comment 3

[Georg Fritzsche [:gfritzsche]]

Alessio, can you give this notebook a sanity check?

Comment 4

[Georg Fritzsche [:gfritzsche]]

I also re-ran my size analysis and it still holds up on per-client volume.
I'd hold back on conclusions though until we have seen a substantial portion of the clients.

Comment 5

[Alessio Placitelli [:Dexter]]

(In reply to Georg Fritzsche [:gfritzsche] from comment #2)
> This is a re-run of the validation from bug 1247605 with some added details:
> https://gist.github.com/georgf/c9f4824e5d3bbf112db7

The notebook looks fine.

Comment 6

[Georg Fritzsche [:gfritzsche]]

I'm dropping this for now until we know more about the release population issue.

Comment 7

[Georg Fritzsche [:gfritzsche]]

We ran this and filed bugs for the problems we are seeing right now.
Closing this, future investigations can go into new bugs.