Closed
Bug 580723
Opened 14 years ago
Closed 14 years ago
hbaseClient.py need an iterator for processed legacy crashes
Categories
(Socorro :: General, task)
Tracking
(Not tracked)
RESOLVED
FIXED
1.8
People
(Reporter: lars, Assigned: dre)
References
Details
we need an iterator that will give a sequence of ooids that are for processed legacy crashes. Should the iterator be perpetual and blocking? Not sure. Probably not.
Reporter | ||
Updated•14 years ago
|
Target Milestone: --- → 1.8
Assignee | ||
Comment 1•14 years ago
|
||
New table added to HBase staging: crash_reports_index_legacy_processed It contains two columns, ids:ooid, and also processed_data:json. The json is a shortcut so the feeder doesn't have to make a separate request to the crash_reports table. We are storing an extra copy of the json, but since life in this queue is short, it shouldn't make a difference. New methods added to hbaseClient in rev 2239: insert_to_legacy_db_feeder_queue is called by put_processed_json. iterator_for_legacy_db_feeder_queue is the method to retrieve items. There are TODO comments in there. One that I didn't add was that you should pass a limit to limited_iteration if you don't want the default(whatever that is) At the moment, iterator_for_legacy_db_feeder_queue calls delete_from_legacy_db_feeder_queue as it retrieves items. That could be called separately if needed.
Assignee | ||
Comment 2•14 years ago
|
||
Also added a new table crash_reports_index_legacy_priority_processed that can be used for a separate queue of priority items so they can always be feed first.
Reporter | ||
Comment 3•14 years ago
|
||
minor point: "crash_reports_index_legacy_priority_processed" does not need to contain the word "legacy". The state of being a priority job has nothing to do with legacy status. A non-legacy crash (a crash that was throttled) can be a priority job.
Assignee | ||
Comment 4•14 years ago
|
||
I think I was abusing the term legacy. I was thinking more about it being code that was only to support 1.8 and going away in 1.9
Assignee | ||
Comment 5•14 years ago
|
||
This was implemented and is in 1.8 staging
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Comment 6•14 years ago
|
||
Not relevant for 1.7.x.
Updated•13 years ago
|
Component: Socorro → General
Product: Webtools → Socorro
You need to log in
before you can comment on or make changes to this bug.
Description
•