Closed Bug 1264330 Opened 8 years ago Closed 8 years ago

Scala derived crash aggregates dataset

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Product:
Cloud Services Graveyard
Component:
Metrics: Pipeline
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: azhang, Assigned: mdoglio)

References

(Depends on 1 open bug)

Details

The current crash aggregates job is written in Python, and takes about 6 hours to run on the ~350M pings per day.

We seem to be CPU bound in some places. Writing a version of the job in Scala [1] alleviates this.

Currently, this version is not ready for primetime. For now, we'll be using the Python job [2], which we've already verified against stability analysis numbers.

[1]: https://github.com/mozilla/telemetry-batch-view/pull/56
[2]: https://github.com/mozilla/moz-crash-rate-aggregates
Assignee: nobody → azhang
Depends on: 1257321
Bumped to p1 as its being currently on by Anthony
Priority: -- → P1
please unassign if this should not find you Mauro
Assignee: azhang → mdoglio
I'm taking care of the deployment of :azhang's patch
This was deployed on 2016/05/18
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.