1637311 - automatically update crash schema when crash_annotations.yaml changes, somehow

Status: NEW

(Data Platform and Tools :: General, task, P4)

Description

[William Lachance (:wlach)]

In bug 1631506, I added a small script for automatically updating the crash schema based on changes in the CrashAnnotations.yaml file. This is better than before, but it is still kind of annoying to do this by hand. No idea how, but it would be great if we could somehow automatically perform these types of updates -- or maybe add some periodic CI to check if something is missing.

Comment 1

[Frank Bertsch [:frank]]

Taking off data-quality tag, this is an improvement to our deploy process.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

Until this is automated, how should someone who is adding a new annotation get the schema updated? I ask because I'm updating the "how to get a new annotation added" docs in Socorro and this is something we should add.

Comment 3

[William Lachance (:wlach)]

(In reply to Will Kahn-Greene [:willkg] ET needinfo? me from comment #2)

Until this is automated, how should someone who is adding a new annotation get the schema updated? I ask because I'm updating the "how to get a new annotation added" docs in Socorro and this is something we should add.

Oh awesome! Happily I just had to go through this procedure so here's a step-by-step set of instructions. Feel free to adjust as desired:

1. From a mozilla-pipeline-schemas (https://github.com/mozilla-services/mozilla-pipeline-schemas/) checkout, run:

scripts/extract_crash_annotation_fields /path/to/mozilla-unified/toolkit/crashreporter/CrashAnnotations.yaml

If any exist, you will get a list of crash annotations that are contained in the ping but are not yet in the schema.

2. Copy them into templates/telemetry/crash/crash.4.schema.json under the payload/metadata section.

3. File a bug to add them under Data Platform and Tools :: Datasets General. Then create a pull request against mozilla-pipeline-schemas referencing that bug.

You can see an example of this here:

https://bugzilla.mozilla.org/show_bug.cgi?id=1652777