Closed
Bug 1435551
Opened 6 years ago
Closed 6 years ago
Remove processed_crashes tables
Categories
(Socorro :: Database, task, P2)
Socorro
Database
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: willkg, Assigned: willkg)
References
Details
Attachments
(1 file)
Socorro stores the processed crash as a JSON blob in the db. However, as near as I can tell, nothing uses that table. This bug covers verifying that's the case and if so, nixing it. That involves: 1. removing saving to the processed crash stable in the Postgres crashstorage classes 2. deleting the ProcessedCrash model 3. deleting all the existing tables (it's partitioned) 4. removing the report_partition_info record 5. removing the code from truncate_partitions stored procedure
Assignee | ||
Comment 1•6 years ago
|
||
Making this a P1. Fixing this reduces the size of our db and makes dump-and-restore a lot easier/faster which will make working on our new infrastructure easier.
Priority: -- → P2
Assignee | ||
Comment 2•6 years ago
|
||
Grabbing this to work on now.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Assignee | ||
Comment 3•6 years ago
|
||
Current table stats as of February 8th, 2018: relation | total_size -----------------------------------+------------ public.processed_crashes_20180129 | 172 GB public.processed_crashes_20180205 | 77 GB public.processed_crashes_20180101 | 272 kB public.processed_crashes_20180115 | 264 kB public.processed_crashes_20180122 | 232 kB public.processed_crashes_20171225 | 184 kB public.processed_crashes_20180108 | 168 kB public.processed_crashes_20180212 | 24 kB public.processed_crashes | 8192 bytes
Assignee | ||
Comment 4•6 years ago
|
||
Assignee | ||
Comment 5•6 years ago
|
||
This should block removing the postgresql crash storage.
Blocks: 1257531
Comment 6•6 years ago
|
||
Commits pushed to master at https://github.com/mozilla-services/socorro https://github.com/mozilla-services/socorro/commit/ce2b0910631d272f075acdc4e4af4a92b9597c93 fix bug 1435551 - stop saving to processed_crashes_* tables We use the PostgreSQLCrashStorage as one of the destinations in the processor. That crash storage class saves raw crash data to the raw_crashes_* tables and processed crash data to a series of reports tables and also the processed_crashes_* tables. However, we no longer look at the data in the processed_crashes_* tables. As such, it's a waste of work, bandwidth, and db use. This nixes that extraneous work. This also merges PostgreSQLBasicCrashStorage with PostgreSQLCrashStorage since it doesn't help us to have two separate classes and all the tests and everything. https://github.com/mozilla-services/socorro/commit/413372252b4d75332df923045e43b4ba7f51290e fixes bug 1435551 - remove processed_crashes* tables This does the following: * deletes the processed_crashes* tables and data * removes the row from the report_partition_info table so we won't be creating new ones * updates the truncate_partitions stored procedure so it doesn't try to truncate tables that don't exist * removes the sqlalchemy model https://github.com/mozilla-services/socorro/commit/d442a4df44a2be121a0ad35a7c3151afc885263a Merge pull request #4332 from willkg/1435551-crash-data-and-processed-crashes fix bug 1299465, 1435551 - remove CrashData classes and stop saving to processed_crashes_* tables
Updated•6 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•