Closed Bug 1182241 Opened 8 years ago Closed 8 years ago

Splice changes to remove table locking on insert

Categories

(Content Services Graveyard :: Tiles: Data Processing, defect)

Product:
Content Services Graveyard
Component:
Tiles: Data Processing
Type:
defect
Priority:
Not set
Severity:
normal
Points:
1

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Iteration:
43.2 - Sep 7

People

(Reporter: oyiptong, Assigned: nanj)

References

(Blocks 2 open bugs)

Details

Redshift necessitated us to use table locking for insertions.
Moving to postgres, we want to remove this locking on insert because it atrociously slows down insertions, causing an ingest to take ~10 mins

More importantly, we need to find the correct lock isolation level for Postgres and test to make sure it is correct
Blocks: Sprint_CS_S2
Iteration: --- → 43.2 - Sep 7
Points: --- → 1
Assignee: nobody → najiang
Commit pushed to master at https://github.com/mozilla/splice

https://github.com/mozilla/splice/commit/c448a109a43caa051d2fa41c3595798e8408b6d0
Close Bug 1182241 - Splice changes to remove table locking on insert

Squashed commit of the following:

commit c80d010dc2a41385a8fab134d27f0ba50a97a673
Merge: 2225d44 4e5e931
Author: Nan Jiang <njiang028@gmail.com>
Date:   Fri Sep 4 14:54:53 2015 -0400

    Merge master to avoid conflicts

commit 2225d44d19d4726c404f1173f95d9906aad44257
Author: Nan Jiang <njiang028@gmail.com>
Date:   Thu Sep 3 15:41:50 2015 -0400

    Enable table locks at the top level

commit dc2fa4d43d4b5f6b6b6728956ac92c4d4400d121
Author: Nan Jiang <njiang028@gmail.com>
Date:   Wed Sep 2 15:54:35 2015 -0400

    Drop the composite unique key

commit 0fcd77a1d33acda3c02c7c24b4577ca9a8ce6c8c
Author: Nan Jiang <njiang028@gmail.com>
Date:   Wed Sep 2 12:04:34 2015 -0400

    Implement the 'all or nothing' scheme while ingesting tiles

commit ed9cff824b93b0c830c83e2827d47aca7efbfabe
Author: Nan Jiang <njiang028@gmail.com>
Date:   Tue Sep 1 16:04:39 2015 -0400

    Add SQLAlchemy to requirements.txt

commit 9c58d2eae8f5465b63816ccd5bc23bc4df81b095
Author: Nan Jiang <njiang028@gmail.com>
Date:   Mon Aug 31 18:32:18 2015 -0400

    Add composite unique key for adgroup and tile

commit 5133b0ebb677173c9342cb00456c0ada5e1c4674
Author: Nan Jiang <njiang028@gmail.com>
Date:   Fri Aug 28 17:04:42 2015 -0400

    Remove table locking

Close #104
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.