921443 - crash reports cannot be fetched and time out

Status: VERIFIED FIXED

(Socorro :: General, task)

Description

[[:philipp]]

recently i have troubles loading fetching crash reports that were provided from users from about:crashes - they will just time out after a few minutes ("Oh Noes! This archived report could not be located.")

these are some sample urls failing:
https://crash-stats.mozilla.com/report/index/72f806e1-7bbf-47ea-aed2-dd4992130926
https://crash-stats.mozilla.com/report/index/0e7b391a-aa4b-4dbf-b189-91fa52130919
https://crash-stats.mozilla.com/report/index/3d1849ba-0800-4d50-a940-e1bc62130925

related discussion going on in the sumo contributor forum: https://support.mozilla.org/en-US/forums/contributors/709646

Comment 1

[Brandon Savage [:brandon]]

I think I know what's going on here.

I'm willing to bet that we're still putting priority jobs into the database, not into RabbitMQ, but with no processors using the database priority queue, the jobs simply persist. :selenamarie can you confirm the size of the priority_jobs table in Postgres?

Comment 2

[Brandon Savage [:brandon]]

I've confirmed that this is in fact the issue. I'm grabbing this bug to fix.

Comment 3

[Brandon Savage [:brandon]]

First pass at resolving this is up: https://github.com/mozilla/socorro/pull/1550

Comment 4

[Selena Deckelmann :selenamarie :selena]

Yup, 1000+ priorityjobs in the database. I'm moving them over to rabbit right now.

Comment 5

[[github robot]]

Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/47f128b7bc83037fb86470cc24c8ade1815399bb
Fixes Bug 921443 - Adds priority jobs for RabbitMQ and changes the Hbase middleware to use the new RabbitMQ priority jobs queue. r=erikrose

Comment 6

[Brandon Savage [:brandon]]

12:54 brandonsavage: Ok, that one didn't process.
12:54 solarce: and i see one in the priority queue!!!