1765414 - Some telemetry uninstall pings missing `type` column

Status: RESOLVED FIXED

(Data Platform and Tools :: General, defect)

Description

[Sean Rose [:srose]]

Weekly platform health check meeting notes indicate this has been a problem going back to at least 2021-05-24.

Comment 1

[Krzysztof Ignasiak [:kik]]

Attachment: Screenshot 2022-07-29 at 12.14.37.png

Adding a screenshot from the explore showing this error:
https://mozilla.cloud.looker.com/merge?mid=BpWTCE6RuxnLkuw2N9XwIv&toggle=pik

Comment 2

[Krzysztof Ignasiak [:kik]]

It appears high-level type property definition was missing inside the schema template for uninstall ping here:
https://github.com/mozilla-services/mozilla-pipeline-schemas/blob/main/templates/telemetry/uninstall/uninstall.4.schema.json#L5

We've worked with Arkadiusz on implementing the change to the template and regenerating the telemetry.uninstall ping schema:
https://github.com/mozilla-services/mozilla-pipeline-schemas/blob/uninstall_ping_missing_type/templates/telemetry/uninstall/uninstall.4.schema.json

We've also noticed that the test definition for this particular schema contained top-level type field, however, the schema tests did not fail to indicate schema misconfiguration. This is something we also attempted to fix and will be part of the same PR.

Change made: https://github.com/mozilla-services/mozilla-pipeline-schemas/commit/2ca3070e03ecc563aff097aef59f4db52d142afb
Test for the schema: https://github.com/mozilla-services/mozilla-pipeline-schemas/blob/main/validation/telemetry/uninstall.4.sample.pass.json#L2

The test now seems to behave the way we expect it to:
https://app.circleci.com/pipelines/github/mozilla-services/mozilla-pipeline-schemas/2848/workflows/3247a6ef-decf-4a4d-9369-88fcdf839ab1/jobs/5477

Comment 3

[Krzysztof Ignasiak [:kik]]

PR opened:

• https://github.com/mozilla-services/mozilla-pipeline-schemas/pull/737

Comment 4

[Krzysztof Ignasiak [:kik]]

PR merged, we should check this graph again on Monday to see if the count of this error starts to reduce. If that's the case this bug can be marked as fixed.

Comment 5

[Arkadiusz Komarzewski [:akomar]]

I think next schema deploy will run on Monday so the effect of this might not yet be visible during the next health check meeting.

Comment 6

[Sean Rose [:srose]]

Should the pings that ended up in the error table because of this issue be backfilled?

Comment 7

[Arkadiusz Komarzewski [:akomar]]

(In reply to Sean Rose from comment #6)

Should the pings that ended up in the error table because of this issue be backfilled?

These pings did not go to the error table. Because json schema allowed additional fields and did not contain type they passed schema validation and type value ended up in additional_properties (see https://docs.telemetry.mozilla.org/concepts/pipeline/schemas.html?highlight=additional_prope#decoding).

I have confirmed that after the schema deploy today telemetry_live.uninstall_v4 has records with type column populated.

Comment 8

[Krzysztof Ignasiak [:kik]]

:party-parrot: