977636 - Use an UPSERT for reports table additions in crashstorage.py

Status: RESOLVED FIXED

(Socorro :: Database, task)

Description

[Selena Deckelmann :selenamarie :selena]

We have a complicated insert or delete and insert for the reports table. See whether we can turn this into an UPSERT and eliminate a few transactions per reprocessed crash.

Comment 1

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/c59383ec00efac17ce815717a8434f049a978231
Fixes bug 977636 - upsert for reports table

https://github.com/mozilla/socorro/commit/cf21d12700978f58ce1c0466b9d2e80f1d84605f
Merge pull request #2158 from selenamarie/bug977636-upsert-for-reports

Fixes bug 977636 - upsert for reports table

Comment 2

[Selena Deckelmann :selenamarie :selena]

Just saw this in the postgres database logs: 

2014-07-01 03:32:03.004 GMT,"processor","breakpad",3948,"[local]",53b22b72.f6c,1,"SELECT",2014-07-01 03:30:58 GMT,6/18469295,778263422,ERROR,23505,"duplicate key value violates unique constraint ""repor
ts_20140630_uuid""",

Unsure how this could happen now.. investigating.

Comment 3

[Selena Deckelmann :selenamarie :selena]

Tested query in the logs and found that the query logged runs without error. Mysterious!

Comment 4

[Selena Deckelmann :selenamarie :selena]

Log from processor7: 

2014-06-30 20:31:59,104 INFO  - Thread-14 - starting job: 6de90b7a-342c-4c18-afd2-097a72140630
2014-06-30 20:31:59,106 INFO  - Thread-5 - starting job: 6de90b7a-342c-4c18-afd2-097a72140630
2014-06-30 20:32:02,483 INFO  - Thread-5 - finishing successful job: 6de90b7a-342c-4c18-afd2-097a72140630
2014-06-30 20:32:02,787 WARNING  - Thread-14 - unable to delete /tmp/6de90b7a-342c-4c18-afd2-097a72140630.upload_file_minidump.TEMPORARY.dump. manual deletion is required.
OSError: [Errno 2] No such file or directory: '/tmp/6de90b7a-342c-4c18-afd2-097a72140630.upload_file_minidump.TEMPORARY.dump'
2014-06-30 20:32:02,804 INFO  - Thread-14 - finishing successful job: 6de90b7a-342c-4c18-afd2-097a72140630
DETAIL:  Key (uuid)=(6de90b7a-342c-4c18-afd2-097a72140630) already exists.
2014-06-30 20:32:04,214 DEBUG  - Thread-5 - mutation size for row_id 61406306de90b7a-342c-4c18-afd2-097a72140630: 343318, execution time: 0:00:00.021009
2014-06-30 20:32:05,051 ERROR  - QueuingThread - RabbitMQCrashStorage tried to acknowledge crash 6de90b7a-342c-4c18-afd2-097a72140630, which was not in the cache
KeyError: '6de90b7a-342c-4c18-afd2-097a72140630'

Comment 5

[Selena Deckelmann :selenamarie :selena]

Ok this bug looks unrelated to my patch, and may be a rabbit issue. Opening a new bug.