1452770 - telemetry-streaming seems to be dropping the display version on many crash pings

Status: RESOLVED DUPLICATE of bug 1452745

(Data Platform and Tools Graveyard :: Datasets: Error Aggregates, enhancement)

Description

[William Lachance (:wlach)]

Compare this count of crash pings and their display versions on April 1st:

https://gist.github.com/wlach/b26b59bd71ece458eeaf886354548863

With these queries on the same day:

* https://sql.telemetry.mozilla.org/queries/52441/source (Mac)
* https://sql.telemetry.mozilla.org/queries/52442/source (Windows)

The results are very strange.

It seems like we're counting the pings without a display version correctly (e.g. for Windows I counted 8492, which matches what I see in the gist). However, it looks like we're vastly undercounting the correctly labeled pings. I count 162 permutations of 60.0b8, but there appear to be 198504 in the dataset as a whole.

What confuses me even more is that the code to process the pings seems very straightforward to me and I don't see why it wouldn't be picking up the displayVersion. Here's the v1a version we're using in production right now:

https://github.com/mozilla/telemetry-streaming/blob/947b24878f27cd62d5b0fdbc60ec74b5ad6c8559/src/main/scala/com/mozilla/telemetry/streaming/ErrorAggregator.scala#L180

:frank, do you have any idea what I might be missing? This (in combination with bug 1452745) is causing our main crash counts on beta to be way off.

Comment 1

[William Lachance (:wlach)]

I was actually just collecting/filtering the data incorrectly in bug 1452745 -- if you exclude non-main pings, the numbers add up to exactly what we'd expect. See that bug for more details.