1595904 - OOMAllocationSize missing from BigQuery crash_v4 table schema

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

crash_summary had a payload.metadata.OOMAllocationSize key. There's no equivalent in the BigQuery crash_v4 table, though it is present in additional_properties. It's only present if a crash was a OOM crash.

Comment 1

[William Lachance (:wlach)]

It does indeed seem to be missing from the schema:

https://github.com/mozilla-services/mozilla-pipeline-schemas/blob/676532f84b2a77a0582432c7f1080cdad51ffbd9/schemas/telemetry/crash/crash.4.schema.json#L1142

May be worth doing a quick pass to see if there are any other fields in the payload which could be trivially added.

Comment 2

[Anthony Miyaguchi [:amiyaguchi]]

There are quite a few fields missing, in particular in the payload.metadata struct. There's a view for enumerating paths in the additional_properties field in the ping tables under moz-fx-data-shared-prod.monitoring.telemetry_missing_columns_v1. Check this redash query for a list of affected fields, including OOMAllocationSize.

Comment 3

[Gabriele Svelto [:gsvelto]]

FYI the fields of a crash ping are extensively documented here:

https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/data/crash-ping.html

Additionally all the subfields of payload.metadata can be extracted from CrashAnnotations.yaml. They are the entries flagged with ping: true.

Comment 4

[Anthony Miyaguchi [:amiyaguchi]]

Attachment: Bug 1595904 - Add missing metadata fields to crash pings from "crash" ping source docs #448

Comment 5

[Tim Smith (inactive) 👨‍🔬 [:tdsmith]]

I think your pull request fixed this, right? (Thanks!)

Comment 6

[Anthony Miyaguchi [:amiyaguchi]]

It did, the field should be available as of 2019-11-14 (https://sql.telemetry.mozilla.org/queries/66646/source).