phc_kind and friends have no storage_mapping
Categories
(Socorro :: Processor, defect, P2)
Tracking
(Not tracked)
People
(Reporter: willkg, Assigned: willkg)
References
Details
Attachments
(1 file)
A while back, I added:
- phc_kind
- phc_base_address
- phc_usable_size
- phc_alloc_stack
- phc_free_stack
When I added them to super_search_fields.py, I didn't set the storage_mapping. Thus they get indexed using whatever ES uses by default. ({"type": "string"}).
We should explicitly set the mapping so that ES isn't inferring things.
|
Assignee | |
Comment 1•3 years ago
|
||
Grabbing this to do now.
|
|
Assignee | |
Comment 2•3 years ago
|
||
Example crash to use: bp-4ec469d3-91f4-4cf6-8d71-fa8760210103
|
|
Assignee | |
Comment 3•3 years ago
|
||
Maybe related... I can't do "phc_kind exists" and get back any crash reports from Crash Stats after like January 14th or so. Maybe I changed something around that time that broke indexing this data? Maybe it's a coincidence with something else?
|
|
Assignee | |
Comment 4•3 years ago
|
||
The last crash report I see with PHCKind annotation is bp-9e4bf0ab-4d7f-40ed-984d-a26850210110 . That shows up before I did my first prod deploy in 2021 (https://github.com/mozilla-services/socorro/releases/tag/2021.01.14). After that prod deploy, searching for "phc_kind exists" brings up nothing.
decoder says it no longer has a maintainer and that he didn't make any changes.
I looked at mozilla-central and at bugs and I don't see any evidence it was disabled. Sure looks like it's still enabled for Linux64 and Win64.
I looked at other PHC fields and found bp-c1a121a4-b95b-44b9-9731-eac1a0210413 .
So if you search for "phc_kind exists", you get nothing, but if you search for "phc_usable_size > 0", you get this one.
phc_usable_size is the only one of the PHC annotations that has a storage mapping defined. So what's going on is that after I pushed the changes to prod on 1/14/2021, the valid keys list is the super search fields which includes phc_kind (that's a different bug) - mapping fields which only has things in the mapping and that doesn't include phc_kind because the mapping is generated from all the fields that define a storage_mapping. Thus phc_kind gets removed from the processed crash before indexing.
I'll tackle fixing the valid keys problem and all that in bug #1706076. In this bug, I'll fix the problem where PHC fields don't define a storage_mapping.
|
|
Assignee | |
Comment 5•3 years ago
|
||
|
|
Assignee | |
Comment 6•3 years ago
|
||
|
|
Assignee | |
Comment 7•3 years ago
|
||
I pushed this to prod today in bug #1708188. However, it won't kick in until Socorro creates a new Elasticsearch index over the weekend. I'll needinfo me to check it then.
|
|
Assignee | |
Comment 8•3 years ago
|
||
I looked at this again today. There's one crash report that I think should show up in a "phc_kind exists" query, but doesn't:
https://crash-stats.mozilla.org/report/index/3707ba36-e346-4a89-8651-882380210502
I can never remember exactly when indexes get created, so it's possible I'm wrong and that was in last week's index.
Anyhow, I can't see what's going on mapping-wise, so I'm going to work on bug #1546800 which will let me see the mapping of the latest index.
|
|
Assignee | |
Comment 9•3 years ago
|
||
I pushed an admin page that shows me everything. The phc_kind field is in the mapping and is getting indexed correctly. The crash report doesn't show up because it's for 2021-05-02 and the new index is created at the beginning of 2021-05-03.
Now that I can verify the mapping and this works on stage, I don't need to wait to verify it. Marking as FIXED.
Description
•