[Baloo] Automating github aggregations from MoFo integration DB

RESOLVED FIXED in 2014Q3

Status

Community Building
Systems and Data
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: pierros, Unassigned)

Tracking

unspecified
2014Q3
Dependency tree / graph

Details

(Whiteboard: [Baloo][Q3][data: ETL])

(Reporter)

Description

3 years ago
Github aggregations need to be automated using the MoFo integration DB.

See 1042057 for details on access.
Working on this now, so far I have gotten:
- data into the raw table
- new contributors into the contributor table
- new repos into the github_repo table
- new github organizations into the github_org table
- set the team for each repo, where applicable, using https://docs.google.com/a/mozilla.com/spreadsheets/d/1cMZQYEmEiuDTAY93j4siiq58exaMZOa2o3YdoLyEYTI/edit#gid=0
- data into the facts table (aggregated from the raw table)
I have integrated the facts table into contributor_facts, and am now going back through to 2010 and doing the weekly active contributor data. Will post here and e-mail when the github numbers are complete and can be seen in Tableau.
The github numbers were complete last week, around 9/2. Things are automated from this table.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Whiteboard: [Baloo][Q3] → [Baloo][Q3][ETL]
Whiteboard: [Baloo][Q3][ETL] → [Baloo][Q3][data: ETL]
Blocks: 1149203
You need to log in before you can comment on or make changes to this bug.