Migrate old data from PostgreSQL into HBase or some other archive

RESOLVED WONTFIX

Status

Mozilla Metrics
Data/Backend Reports
RESOLVED WONTFIX
8 years ago
6 years ago

People

(Reporter: laura, Unassigned)

Tracking

unspecified
Unreviewed
x86
Mac OS X

Details

(Reporter)

Description

8 years ago
We'd like to migrate old data out of PostgreSQL.  We have stuff going back to 2008 which can't be queried from the webapp.  To make this available it should migrate into HBase.
Would it be possible to dump this data in a plain text format and tarball it in some fashion?  Would make it easier to put in cold storage if we don't have to slurp directly from Postgres.
We can clone the current production database into dm-breakpad-devdb, and you can massage it into whatever format works best for you.  Dumping it all out to plain text may not be possible (afaik, we have old archived reports in there?)
We discussed this bug in triage today and came up with an alternate solution for which we are curious to get feedback regarding viability.
It is to write a simple python script that queried a Postgres DB for all the old processed crashes and extracts the processed json and uses the existing hbaseClient.py call to insert the data into the HBase processed_data:json cell.

Thoughts?

Comment 4

6 years ago
CLosing all old (created < 1-1-2011 ) cases still in NEW state.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.