1713355 - Make "is" searches work on 'mac_crash_info'

Status: RESOLVED FIXED

(Socorro :: General, defect, P2)

Description

[Steven Michaud [:smichaud] (Retired)]

Currently (at crash-stats.mozilla.org) an "is" search on "mac crash info" returns no results, even if matching crash reports do exist:

https://crash-stats.mozilla.org/search/?mac_crash_info=%3D%7B%20%20%22num_records%22%3A%202%2C%20%20%22records%22%3A%20%5B%20%20%20%20%7B%20%20%20%20%20%20%22message%22%3A%20%22abort%28%29%20called%22%2C%20%20%20%20%20%20%22module%22%3A%20%22%2Fusr%2Flib%2Fsystem%2Flibsystem_c.dylib%22%20%20%20%20%7D%2C%20%20%20%20%7B%20%20%20%20%20%20%22module%22%3A%20%22%2FSystem%2FLibrary%2FPrivateFrameworks%2FGPUSupport.framework%2FVersions%2FA%2FLibraries%2FlibGPUSupportMercury.dylib%22%2C%20%20%20%20%20%20%22signature_string%22%3A%20%22Graphics%20kernel%20error%3A%200x1be385f9%5Cn%22%20%20%20%20%7D%20%20%5D%7D&version=90.0a1&platform=Mac%20OS%20X&date=%3E%3D2021-04-28T16%3A31%3A00.000Z&date=%3C2021-05-28T16%3A31%3A00.000Z&_facets=signature&_facets=mac_crash_info&page=1&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform

https://crash-stats.mozilla.org/search/?mac_crash_info=~0x1be385f9&date=%3E%3D2021-05-21T16%3A44%3A00.000Z&date=%3C2021-05-28T16%3A44%3A00.000Z&_facets=signature&_facets=mac_crash_info&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-mac_crash_info

A "contains" search also fails if the search criterion is the entire "mac crash info" record:

https://crash-stats.mozilla.org/search/?mac_crash_info=~%7B%20%20%22num_records%22%3A%202%2C%20%20%22records%22%3A%20%5B%20%20%20%20%7B%20%20%20%20%20%20%22message%22%3A%20%22abort%28%29%20called%22%2C%20%20%20%20%20%20%22module%22%3A%20%22%2Fusr%2Flib%2Fsystem%2Flibsystem_c.dylib%22%20%20%20%20%7D%2C%20%20%20%20%7B%20%20%20%20%20%20%22module%22%3A%20%22%2FSystem%2FLibrary%2FPrivateFrameworks%2FGPUSupport.framework%2FVersions%2FA%2FLibraries%2FlibGPUSupportMercury.dylib%22%2C%20%20%20%20%20%20%22signature_string%22%3A%20%22Graphics%20kernel%20error%3A%200x1be385f9%5Cn%22%20%20%20%20%7D%20%20%5D%7D&version=90.0a1&platform=Mac%20OS%20X&date=%3E%3D2021-04-28T16%3A31%3A00.000Z&date=%3C2021-05-28T16%3A31%3A00.000Z&_facets=signature&_facets=mac_crash_info&page=1&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform

Can we make these searches work? Otherwise it won't be possible to search for crash reports by clicking on an item in a list "faceted" on "mac crash info".

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

The "is" query does a term filter against the ".full" value. I'm not sure what's going on. I'll have to look into it.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I tinkered with this a little today. My current hunch is that this is a problem with whitespace matching. When the processor extracts the value from json_dump, it encodes the structure as JSON in a way that's good for display, but probably bad for indexing and searching.

I'll test out my hunch tomorrow.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

I verified it's a problem with the whitespace. Also, there's this annoying thing where if you're going from a "mac_crash_info exists" supersearch and then click on a value to refine the search, you end up with a supersearch doing "mac_crash_info exists" and "mac_crash_info is <somevalue>". When that happens, supersearch is doing an OR between those filters because it's for the same field--you're getting "mac_crash_info exists OR mac_crash_info is <somevalue>". That's unhelpful.

I'm going to fix "is" for mac_crash_info by removing the whitespace in the indexed value. Then what you're doing will mostly work. You'll need to manually remove the "mac_crash_info exists" filter from the refined query.

I'm going to defer fixing the "we shouldn't mix exists with is for the same field" issue to a new bug.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 5971: bug 1713355: fix mac_crash_info "is" searches

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #5971: "bug 1713355: fix mac_crash_info "is" searches" in a5df220.

Comment 6

[Will Kahn-Greene [:willkg] ET needinfo? me]

This deployed a little bit ago in bug #1749582.

I'm still working on a fix for bug #1749561. Also, this changes how the data is extracted from json_dump, so we'd need to reprocess affected crash reports. It looks like there are a fair number of crash reports with mac_crash_info.

Steven: I can reprocess crash reports. How far back do you want me to reprocess?

Comment 7

[Steven Michaud [:smichaud] (Retired)]

Steven: I can reprocess crash reports. How far back do you want me to reprocess?

Part of me wants to say "all of them" -- in other words go back six months.

Is that reasonable? If not, then I leave the decision up to you.

(Of course you'd only need to reprocess reports that actually contain (non-empty) mac_crash_info.)

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

I reprocessed all the crash reports with mac_crash_info. Marking as FIXED.

Comment 9

[Steven Michaud [:smichaud] (Retired)]

I did a quick test of this, by doing a "mac_crash_info exists" search, faceting on mac_crash_info, then right-clicking on one of the "facets". I did have to remove the original "mac_crash_info exists" line (now redundant, bug 1749561). But afterwards everything was as expected.

Thanks!