Running the CrashAggregates test suite requires 4GB of RAM dedicated to the JVM. It's not clear why this is happening, but it seems to have something to do with the increasing number of dimensions in the test fixtures.
Note to myself: also add a testcase for pings with missing dimensions.
Component: Metrics: Pipeline → Datasets: Crash Aggregates
Product: Cloud Services → Data Platform and Tools
My guess is it's the fact that we're generating a unique ping for every possible combination of these dimensions: https://github.com/mozilla/telemetry-batch-view/blob/4f1563d/src/test/scala/com/mozilla/telemetry/CrashAggregateViewTest.scala#L15 https://github.com/mozilla/telemetry-batch-view/blob/4f1563d/src/test/scala/com/mozilla/telemetry/CrashAggregateViewTest.scala#L154 That adds up to 98304 pings. Depending on the exact size of each ping, we could easily get into hundreds of megabytes or even gigabytes (2k per ping = 200M, 20k per ping = 2G) Do we really need every combination here for the test to be valid?
I don't think we do. I would put this bug on hold for now, as bug 1357793 may make this one a WONTFIX.
Component: Datasets: Crash Aggregates → Datasets: General
You need to log in before you can comment on or make changes to this bug.