1368498 - Rewrite bugzilla cron job to not write to the `bugs` table

Status: RESOLVED FIXED

(Socorro :: Backend, task)

Description

[[DEACTIVATED] Adrian Gaudebert]

That data isn't used anymore.

Comment 1

[[DEACTIVATED] Adrian Gaudebert]

Attachment: Link to Github pull-request: https://github.com/mozilla-services/socorro/pull/3804

Comment 2

[[github robot]]

Commit pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/1e8bc808e2bc2907ad5775159b107b28507488ba
Fixes bug 1368498 - Do not use bugs table in bugzilla cron job. (#3804)

* Fixes bug 1368498 - Do not use bugs table in bugzilla cron job.
This makes the bugzilla cron job stop writing any data into the bugs table. It also removes the foreign key from bug_associations to bugs in a migration, which is needed for the cron job to keep working.
It also removes the call to the ``reports`` table, making the job write all associations without verifying if the signatures exist in our system. This happened because 1. we want to get rid of the reports table and 2. I believe this was a source of bugs (with signatures incorrectly missing in that table).

* Some cosmetic changes, and removed useless tests.

* Removed unneeded data from bugzilla.

* Made tests more robust.

Comment 3

[[DEACTIVATED] Adrian Gaudebert]

Confirmed working with bug 1372243 and associated signature https://crash-stats.mozilla.com/signature/?signature=libX11.so.6.3.0%400x41d5b

Comment 4

[[DEACTIVATED] Adrian Gaudebert]

Guess what! Code hadn't been pushed to the stage admin node, so I wasn't actually verifying anything. Now that Will has updated the node, we do see an error: https://sentry.prod.mozaws.net/operations/socorro-stage/issues/615991/

That happens because there's a migration we need to run. I tried to do that, and it failed because, for some mystical reason, the name of the constraint I am trying to remove in that migration is different on stage / prod than it was in my local database. I need to update that migration, and re-run it on stage.

Comment 5

[[DEACTIVATED] Adrian Gaudebert]

Attachment: Link to Github pull-request

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/017b7920ac854980e1a3c2d0177e7ebfb625fdff
Fixes bug 1368498 - Removed correct foreign key from bug_associations table. (#3821)

Comment 7

[[DEACTIVATED] Adrian Gaudebert]

The migration worked, but now running the job sometimes triggers a KeyError: https://sentry.prod.mozaws.net/operations/socorro-stage/issues/616023/

KeyError: 'cf_crash_signature'
  File "crontabber/app.py", line 1053, in _run_one
    for last_success in self._run_job(job_class, config, info):
  File "crontabber/base.py", line 189, in main
    function()
  File "crontabber/base.py", line 259, in _run_proxy
    return self.run(*args, **kwargs)
  File "socorro/cron/jobs/bugzilla.py", line 103, in run
    ) in self._iterator(last_run_formatted):
  File "socorro/cron/jobs/bugzilla.py", line 187, in _iterator
    find_signatures(report['cf_crash_signature'])

This is easy to fix.

Comment 8

[[DEACTIVATED] Adrian Gaudebert]

Attachment: Link to Github pull-request

Comment 9

[[github robot]]

Commit pushed to master at https://github.com/mozilla-services/socorro

https://github.com/mozilla-services/socorro/commit/d81893711bc1a5943d6cf28c249e5613279550a8
Fixes bug 1368498 - Do not error out if crash signature field is missing. (#3823)