1768855 - is_garbage_collecting is the wrong type and kicks up errors in telemetry crashstorage

Status: RESOLVED FIXED

(Socorro :: Processor, defect, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

When the json schema reducer reduces a processed crash using the telemetry_socorro_crash.json schema to generate a crash document destined for Telemetry, it dies on the is_garbage_collecting value.

Sentry: https://sentry.prod.mozaws.net/operations/socorro-new-prod/issues/19480180/

Traceback:

InvalidDocumentError: invalid: .is_garbage_collecting: type not in ['string', 'null']
  File "socorro/external/crashstorage_base.py", line 478, in save_processed_crash
    a_store.save_processed_crash(raw_crash, processed_crash)
  File "socorro/external/crashstorage_base.py", line 639, in benchmarker
    return wrapped_attr(*args, **kwargs)
  File "socorro/external/boto/crashstorage.py", line 309, in save_processed_crash
    crash_report = schema_reduce(
  File "socorro/lib/libjson.py", line 241, in schema_reduce
    return reducer.traverse(schema_part=schema, document_part=document)
  File "socorro/lib/libjson.py", line 202, in traverse
    new_doc[name] = self.traverse(
  File "socorro/lib/libjson.py", line 140, in traverse
    raise InvalidDocumentError(

The value is a Python True which is a boolean. Seems like this field is either True or not there at all. The schema specifies valid types are [string, null].

We can't change the schema, so we're going to have to add a manual fix for historical reasons.

What are the values currently in telemetry.socorro_crash? I suspect it's saving str(value) which would be "True".

Once we figure that out, we can figure out how to change the type with the current system. Maybe converters are something we should build in because we might need more of them?

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Example crash reports:

• bp-7c71d891-c64d-4ec0-9da3-5d4ff0220511
• bp-02c1ed3a-1121-4bb8-b8aa-59b6e0220511

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I was wrong--the crash ping data shows either is_garbage_collecting is not there (e.g. null) or it's the string "1" which is the value from the crash annotation.

Also, I already had this problem and worked out a solution and we have converters, so this is a one line change.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6089: bug 1768855: fix is_garbage_collecting in telemetry crash storage

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6089: "bug 1768855: fix is_garbage_collecting in telemetry crash storage" in 9804d0a.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I pushed this to prod just now in bug #1769264. I reprocessed the two crash reports. Marking as FIXED.