1619637 - [traceback] "TypeErrorconfigman.dotdict in assign"

Status: RESOLVED FIXED

(Socorro :: Processor, defect, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

Sentry: https://sentry.prod.mozaws.net/operations/socorro-new-prod/issues/6794800/

Traceback:

TypeError: list indices must be integers or slices, not str
  File "processor_app.py", line 171, in _transform
    raw_crash = self.source.get_raw_crash(crash_id)
  File "socorro/external/crashstorage_base.py", line 626, in get_raw_crash
    result = self.wrapped_crashstore.get_raw_crash(crash_id)
  File "socorro/external/boto/crashstorage.py", line 213, in get_raw_crash
    raw_crash_as_string, object_hook=self.config.json_object_hook
  File "__init__.py", line 361, in loads
    return cls(**kw).decode(s)
  File "json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "configman/dotdict.py", line 112, in __init__
    self[key] = value
  File "configman/dotdict.py", line 164, in __setitem__
    self.assign(key, value)
  File "configman/dotdict.py", line 236, in assign
    cur_dict[key_split[-1]] = value

Looks like the processor is having problems loading some raw crashes. It affected 300 or so crashes in the last 3 months.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Making this a P2 to look into. 300 in 3 months isn't a big deal, but it's a troubling problem.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

This is preventing some crash reports from being processed. Putting this in my queue to look into.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Here's a crash report it's dying on: e1c2d359-86e7-406b-be6e-2088e0200316

The processor tries to fetch the raw crash from S3. It's in JSON, so it decodes it using configman as the object hook so the processor ends up with a DotDict. configman chokes on this:

'BullGuard Ltd.': ['BgAMSI.dll', 'BackupShellHook.dll'], 

This is in the value for ModuleSignatureInfo which recently switched from having a JSON-encoded string as the value to having an object value.

The key has a . which configman thinks means there's a hierarchy, but the second part is an empty string and this shouldn't be a hierarchy in the first place and everything is a mess.

Bug #1607806 added a rule to convert the value of ModuleSignatureInfo to a JSON-encoded string, but in order for that rule to run, the raw crash report has to load first.

Bug #1607804 covers fixing this in Firefox crash reporter. That took effect in Firefox 74. The crash reports we're having problems with are 73.0.1. We'll need to support them for the foreseeable future, so we should fix this.

Bug #1383113 covers rewriting the processor to use getitem notation. After that's done, we can get rid of configman DotDict and the above problem goes away (possibly to be replaced with different problems). I don't think I'm going to get to this anytime soon, so I think we need an interim solution.

The json_object_hook is set to configman.dotdict.DotDict. Maybe we can write a different object hook that handles this better?

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

I don't think I can easily write a different object hook without creating a new set of problems.

I think the best way to fix this is to fix bug #1383113 and stop using DotDict for crash data. Going to make this bug depend on that one.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'd like to get to this, but it's tricky. Going to push this off for a bit.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

I checked Sentry just now and the Sentry error is no longer available, so I think we haven't seen one of these in 6 months.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

I haven't seen one of these in a while, so I'm going to mark this as FIXED.