similar to #834403 All collectors have slowed to 20-25 crashes pushed into HBase per minute.
This bug triggered investigation that revealed we had a lot of crashes backed up on collector05 and then multiple collectors were very slow: https://bugzilla.mozilla.org/show_bug.cgi?id=836619
Need to run backfill for 1/30 once crashmover catches up
Collectors and crash movers have once again been rolled back to Socorro version 33 (as we did in bug 834403, which makes them fast again. Lars is attempting to reproduce in stage. Problem appears to be a crash mover performance issue of some kind, as we are not seeing HBase errors. Once crash movers are caught up, selena will run backfill.
Collectors are all caught up. :)
Waiting to run backfill until the replica is done being recreated.
4 years ago
Backfill was run long ago!