Closed Bug 1493768 Opened 4 years ago Closed 4 years ago

move "bug_associations" table to django

Categories

(Socorro :: Database, task, P2)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

References

Details

Attachments

(5 files)

Socorro maintains a table associating bug ids from Bugzilla to crash signatures. This table is currently maintained by sqlalchemy and alembic. It's fed by the bugzilla crontabber app. It's used by the webapp.

This bug covers re-creating that table in Django, moving all the data over, updating the bugzilla crontabber app and webapp to use the new version, and removing the old one.
This should be pretty straight-forward since this table isn't tangled up with anything else.
Priority: -- → P2
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Component: Webapp → Database
Summary: move bug_associations table to django → move "bug_associations" table to django
Things we need to do here:

1. create a Django model, Django migration, and Django model admin
2. write a data migration to copy from the old table to the new one
3. reimplement the BugzillaCronApp to update the new tables and the old tables
4. update tests and write any needed new ones

Push that out and verify the data got migrated correctly and BugzillaCronApp is updating data in both places.

4. reimplement the /api/Bugs and /api/SignaturesByBugs endpoints using the Django model and rip out all the related middleware bits
5. update report_index view to use the new Django model
6. update exploitability_report view to use the new Django model
7. update supersearch view to use the new Django model
8. update signature view to use the new Django model
9. update topcrashers view to use the new Django model
10. update tests and write any needed new ones

Push that out and verify it's working.

7. fix BugzillaCronApp to only update the new table
8. update tests and write any needed new ones
9. write a migration to delete the old table
Comment #2 involves some interesting counting. Oops.
Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/e2c6197a6779a7771d8736dd936bae5a5d86ee78
bug 1493768: create BugAssociation Django model

This creates a BugAssociation Django model and updates the bugzilla
cron job to save data to both the old and new tables.

This also adds a data migration script.

This also removes test_run_job_based_on_last_success. After we removed
the bugs table a couple of years ago, this test was adjusted and the
bugs table bits removed. As near as I can tell, this doesn't test anything
useful anymore.

https://github.com/mozilla-services/socorro/commit/c4bea61d62f50f30326c1c92d0ec004aa67da185
Merge pull request #4641 from willkg/1493768-bugs-step-1

bug 1493768: create BugAssociation Django model
Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/6636045a3db64d047b163e5539cb68a1e9f6b3e0
bug 1493768: redo webapp to use BugAssociation Django model

This redoes the webapp to use the BugAssociation Django model instead of
the old Bugs middleware model.

There are a few things to point out.

First, the Bugs middleware model did this funny thing where if you passed
in signatures, then it would find all the bugs associated with those
signatures and then find all the signatures associated with all those bugs.
I redid this as .get_bugs_and_related_bugs().

Second, some of the code that used the old Bugs middleware model relied on
this behavior. I redid that using .get_bugs_and_related_bugs(). Some of
the code filtered the results down to just the (bug, signature) sets for
signatures it asked for. I redid that code using just .filter().

Third, I changed the shape of things, so I had to change templates to
look at the bug_id field instead of the id field.

This does not remove the old table, but it does remove all use of it.

https://github.com/mozilla-services/socorro/commit/57d92ee163c6cf5ac9e8254c8dbf49336ce45320
Merge pull request #4642 from willkg/1493768-ba-step-2

bug 1493768: redo webapp to use BugAssociation Django model
Commits pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/74f773c5414221e564685b8ff321a53bb72cb903
bug 1493768: remove old model and references to it

https://github.com/mozilla-services/socorro/commit/26546046d8866607874969f6b00ace96b50c29ef
Merge pull request #4644 from willkg/1493768-remove-old-model

bug 1493768: remove old model and references to it
I created a new bug to delete tables. Marking this one as FIXED.
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.