Closed Bug 1499438 Opened 1 year ago Closed 1 year ago

Backfill missing data in m-c due to kafka outage

Categories

(Data Platform and Tools :: Operations, enhancement)

enhancement
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: wlach, Unassigned)

References

Details

Attachments

(1 file)

Now that :akomar has filled the missing data in error_aggregates due to bug 1496597, we should also backfill the missioncontrol production service. I am currently backfilling the stage db, and it seems to be working fine (a little slow due to the way we're interacting with the db, but that's ok).

My procedure for doing this (feel free to modify as you see fit):

* ssh to the machine running the m-c worker docker container
* run `screen` (so we can leave the process running uninterrupted)
* docker exec -it <container> /bin/bash
* ./manage.py load_measure_data --start-date 2018-10-03 --end-date 2018-10-05

This depends on bug 1499433 being solved first, because the last command depends on some changes in m-c v1.5.
Flags: needinfo?(whd)
This is running.
Flags: needinfo?(whd)
Attached file traceback.txt
See the attached traceback from executing the command.
I must admit I haven't seen that error before (CC'ing akomar and mreid, to see if it rings a bell for them). I suspect this might just be an intermittent issue, can you try again?
The rerun exited successfully. :wlach if you can verify things look good we can close this out.
Gave the windows data a close look on all channels, seems like the problem is fixed. Thank you!!
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.