1714190 - mac_crash_info "contains" search is case-sensitive

Status: RESOLVED FIXED

(Socorro :: General, defect)

Description

[Steven Michaud [:smichaud] (Retired)]

This may be intended behavior, and not a bug. But it's annoying when searching in a large, free-form field like "mac crash info".

Here's a search on "Graphics hardware" that works:

https://crash-stats.mozilla.org/search/?mac_crash_info=~Graphics%20hardware&date=%3E%3D2021-05-02T19%3A38%3A00.000Z&date=%3C2021-06-02T19%3A38%3A00.000Z&_facets=signature&_facets=platform_version&_facets=mac_crash_info&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-mac_crash_info

And one on "graphics hardware" that doesn't:

https://crash-stats.mozilla.org/search/?mac_crash_info=~graphics%20hardware&date=%3E%3D2021-05-02T19%3A38%3A00.000Z&date=%3C2021-06-02T19%3A38%3A00.000Z&_facets=signature&_facets=platform_version&_facets=mac_crash_info&page=1&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-mac_crash_info

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Pretty sure "contains" is doing a wildcard query, so it's doing strict string matching.

• https://github.com/mozilla-services/socorro/blob/7a82dcee5f88c25ec03c4e1e3afcbc10fa019772/socorro/external/es/supersearch.py#L326-L337
• https://www.elastic.co/guide/en/elasticsearch/reference/1.4/query-dsl-wildcard-query.html

Sorry it's annoying! It'd really help if you were more specific about what kinds of questions you're trying to answer. I thought I had covered everything you were looking for in comment 18, but apparently not?

As an aside, I'm on PTO until mid-July, so I'm not going to be able to make any changes until then. Let me know more about what you're trying to do, what questions you're trying to answer, how you're trying to use search, and then maybe we can figure out what changes we need to make to facilitate that.

Comment 2

[Steven Michaud [:smichaud] (Retired)]

It's not a big deal. The other bug 1709658 and bug 1577886 followup bugs I've opened (bug 1713355 and bug 1711956) are much more important.

The reason I find it annoying is that, when the "mac crash info" field is large, I can forget the exact case of what I'm searching on. But I can learn to live with it. And case-sensitive searches might sometimes be necessary.

Would it be possible to have two different "contains" searches, one case-sensitive and one case-insensitive?

Enjoy your PTO! If I notice a bug that's truly urgent, I'll try to find someone else to work on it.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

If you switch to "has terms", then it does a search using the analyzed field which tokenizes the value and lower cases the tokens.

For example, this now works fine:

"mac_crash_info has terms 'Graphics'":

https://crash-stats.mozilla.org/search/?mac_crash_info=Graphics&date=%3E%3D2022-01-03T21%3A55%3A00.000Z&date=%3C2022-01-10T21%3A55%3A00.000Z&_facets=signature&_facets=platform_version&_facets=mac_crash_info&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports

"mac_crash_info" has terms 'graphics'":

https://crash-stats.mozilla.org/search/?mac_crash_info=graphics&date=%3E%3D2022-01-03T21%3A55%3A00.000Z&date=%3C2022-01-10T21%3A55%3A00.000Z&_facets=signature&_facets=platform_version&_facets=mac_crash_info&_sort=-date&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#crash-reports

"contains" is a strict substring search against the non-analyzed field value, so it will always be case-sensitive.

The docs for this are terrible and I keep forgetting this. I'll fix them now.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 5970: bug 1714190: minor improvement to supersearch docs

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #5970: "bug 1714190: minor improvement to supersearch docs" in a8509fe.

Comment 6

[Steven Michaud [:smichaud] (Retired)]

Thanks. I consider this issue fixed.

I'll put something in User Story to help others who bump into this issue.