558245 - Modify HBase Thrift connection to use new row key format

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Daniel Einspanjer [:dre] [:deinspanjer]]

As per the e-mail I sent to socorro-dev, I feel it would be best for us to change the HBase schema to store the OOID without prefixing the date.

It should be fairly straightforward to change in the code, but I'm happy to help with a review or patch if need be.

Comment 1

[K Lars Lohn [:lars] [:klohn]]

well, in the latest version of hbaseClient.py I just happened to refactor that into a single function call rather than having it spread like buckshot through the code.  The change will be trival.

Of course, we'll have to take all the exsting data within hbase and "correct" it, ja?  That will have to be a coordinated action. 

How soon do you want to make this change?

Comment 2

[Daniel Einspanjer [:dre] [:deinspanjer]]

hehe.. buckshot.

My thoughts are to create a new table (but what better name than "crash_reports" could we possibly come up with? and start inserting the new format into that.  Then, we can easily have a MapReduce job that will take all the data from the old table and fix the row keys as it inserts into the new table.  Bonus, we can do the json fix at the same time.

Comment 3

[Daniel Einspanjer [:dre] [:deinspanjer]]

The change described in the summary was done.
My thoughts in comment #2 are now expressed in other more specific bugs.