Closed Bug 1182241 Opened 10 years ago Closed 9 years ago

Splice changes to remove table locking on insert

Categories

(Content Services Graveyard :: Tiles: Data Processing, defect)

defect
Not set
normal
Points:
1

Tracking

(Not tracked)

RESOLVED FIXED
Iteration:
43.2 - Sep 7

People

(Reporter: oyiptong, Assigned: nanj)

References

Details

Redshift necessitated us to use table locking for insertions. Moving to postgres, we want to remove this locking on insert because it atrociously slows down insertions, causing an ingest to take ~10 mins More importantly, we need to find the correct lock isolation level for Postgres and test to make sure it is correct
Blocks: Sprint_CS_S2
Iteration: --- → 43.2 - Sep 7
Points: --- → 1
Assignee: nobody → najiang
Commit pushed to master at https://github.com/mozilla/splice https://github.com/mozilla/splice/commit/c448a109a43caa051d2fa41c3595798e8408b6d0 Close Bug 1182241 - Splice changes to remove table locking on insert Squashed commit of the following: commit c80d010dc2a41385a8fab134d27f0ba50a97a673 Merge: 2225d44 4e5e931 Author: Nan Jiang <njiang028@gmail.com> Date: Fri Sep 4 14:54:53 2015 -0400 Merge master to avoid conflicts commit 2225d44d19d4726c404f1173f95d9906aad44257 Author: Nan Jiang <njiang028@gmail.com> Date: Thu Sep 3 15:41:50 2015 -0400 Enable table locks at the top level commit dc2fa4d43d4b5f6b6b6728956ac92c4d4400d121 Author: Nan Jiang <njiang028@gmail.com> Date: Wed Sep 2 15:54:35 2015 -0400 Drop the composite unique key commit 0fcd77a1d33acda3c02c7c24b4577ca9a8ce6c8c Author: Nan Jiang <njiang028@gmail.com> Date: Wed Sep 2 12:04:34 2015 -0400 Implement the 'all or nothing' scheme while ingesting tiles commit ed9cff824b93b0c830c83e2827d47aca7efbfabe Author: Nan Jiang <njiang028@gmail.com> Date: Tue Sep 1 16:04:39 2015 -0400 Add SQLAlchemy to requirements.txt commit 9c58d2eae8f5465b63816ccd5bc23bc4df81b095 Author: Nan Jiang <njiang028@gmail.com> Date: Mon Aug 31 18:32:18 2015 -0400 Add composite unique key for adgroup and tile commit 5133b0ebb677173c9342cb00456c0ada5e1c4674 Author: Nan Jiang <njiang028@gmail.com> Date: Fri Aug 28 17:04:42 2015 -0400 Remove table locking Close #104
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.