1635212 - telemetry.crash: Missing `payload.stackTraces.threads` data due to incomplete schema

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect, P2)

Description

[William Lachance (:wlach)]

The missing columns dashboard shows that we are missing a large number of entries in payload.stackTraces.threads for telemetry.crash

Based on investigation by :ascholtz, it looks like this array can contain "frame" objects, which are not currently being processed. This is my mistake, I should have (at least temporarily) enabled strict validation of the schema to validate that I was indeed processing all the fields.

Comment 1

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/541

Comment 2

[Anna Scholtz [:ascholtz]]

I also noticed AvailableSwapMemory and PurgeablePhysicalMemory as missing fields for some telemetry.crash entries.
They show up in additional_properties as {"payload":{"metadata":{"AvailableSwapMemory":"0","PurgeablePhysicalMemory":"810921984"}}}

Might be worth opening a separate bug for those?

Comment 3

[Anna Scholtz [:ascholtz]]

Ah, I see there already exists a bug for this: https://bugzilla.mozilla.org/show_bug.cgi?id=1631506

Comment 4

[William Lachance (:wlach)]

(In reply to Anna Scholtz from comment #3)

Ah, I see there already exists a bug for this: https://bugzilla.mozilla.org/show_bug.cgi?id=1631506

Yep, currently only mentions AvailableSwapMemory, but let's handle PurgeablePhysicalMemory there.

Comment 5

[William Lachance (:wlach)]

Thanks to Frank for reviewing the PR, filed bug 1636239 as a followup, gonna close this as the most pressing issue is fixed (or at least will be, once we do a schema deploy)