fix TelemetryBotoS3CrashStorage so it doesn't use super_search_fields
Categories
(Socorro :: Processor, task, P2)
Tracking
(Not tracked)
People
(Reporter: willkg, Assigned: willkg)
References
Details
Attachments
(3 files)
super_search_fields is (ab)used as a processed crash schema, but it really isn't that--it's the index/query instructions for crash data for Elasticsearch. Using it anywhere else is bad.
Once bug #1753521 is done, then all the crash data we want to send to BigQuery is in the processed crash with the right name and we don't need to use super_search_fields anymore in the save_processed_crash method.
This bug covers fixing TelemetryBotoS3CrashStorage accordingly.
|
Assignee | |
Updated•2 years ago
|
|
|
Assignee | |
Comment 1•2 years ago
|
||
|
|
Assignee | |
Comment 2•2 years ago
|
||
willkg merged PR #6052: "bug 1755095: fix TelemetryBotoS3CrashStorage to not use super search fields" in 129d37b.
I'm going to let this run on stage over the weekend and see how things go. I did test with 1,000 crash reports in my local dev environment, but testing on stage will be more comprehensive.
|
|
Assignee | |
Comment 3•2 years ago
|
||
Oops--I need to fix validate_telemetry_socorro_crash.py to not request/use the RawCrash. It doesn't need it anymore.
|
|
Assignee | |
Comment 4•2 years ago
|
||
|
|
Assignee | |
Comment 5•2 years ago
|
||
|
|
Assignee | |
Comment 6•2 years ago
|
||
From stage:
invalid: .ipc_system_error: type not in ['string', 'null']
The value there is 6. I think we need to add a socorro_convert_to for this.
|
|
Assignee | |
Comment 7•2 years ago
|
||
|
|
Assignee | |
Comment 8•2 years ago
|
||
|
|
Assignee | |
Comment 9•2 years ago
|
||
This went to production in bug #1763234. I'll check the ingestion job tomorrow to see how things went.
|
|
Assignee | |
Comment 10•2 years ago
|
||
The ingestion job did fine. I verified that there's data in telemetry.socorro_crash that was ingested by that job and the data looks fine. Marking as FIXED.
Description
•