we need an iterator that will give a sequence of ooids that are for processed legacy crashes. Should the iterator be perpetual and blocking? Not sure. Probably not.
Assignee: nobody → deinspanjer
New table added to HBase staging: crash_reports_index_legacy_processed It contains two columns, ids:ooid, and also processed_data:json. The json is a shortcut so the feeder doesn't have to make a separate request to the crash_reports table. We are storing an extra copy of the json, but since life in this queue is short, it shouldn't make a difference. New methods added to hbaseClient in rev 2239: insert_to_legacy_db_feeder_queue is called by put_processed_json. iterator_for_legacy_db_feeder_queue is the method to retrieve items. There are TODO comments in there. One that I didn't add was that you should pass a limit to limited_iteration if you don't want the default(whatever that is) At the moment, iterator_for_legacy_db_feeder_queue calls delete_from_legacy_db_feeder_queue as it retrieves items. That could be called separately if needed.
Also added a new table crash_reports_index_legacy_priority_processed that can be used for a separate queue of priority items so they can always be feed first.
minor point: "crash_reports_index_legacy_priority_processed" does not need to contain the word "legacy". The state of being a priority job has nothing to do with legacy status. A non-legacy crash (a crash that was throttled) can be a priority job.
I think I was abusing the term legacy. I was thinking more about it being code that was only to support 1.8 and going away in 1.9
This was implemented and is in 1.8 staging
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Not relevant for 1.7.x.
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.