1309214 - refactor CrashAggregateViewTest to reduce the memory usage

Status: RESOLVED WONTFIX

(Data Platform and Tools :: General, defect, P2)

Description

[Mauro Doglio [:mdoglio]]

Running the CrashAggregates test suite requires 4GB of RAM dedicated to the JVM. It's not clear why this is happening, but it seems to have something to do with the increasing number of dimensions in the test fixtures.

Comment 1

[Mauro Doglio [:mdoglio]]

Note to myself: also add a testcase for pings with missing dimensions.

Comment 2

[William Lachance (:wlach)]

My guess is it's the fact that we're generating a unique ping for every possible combination of these dimensions:

https://github.com/mozilla/telemetry-batch-view/blob/4f1563d/src/test/scala/com/mozilla/telemetry/CrashAggregateViewTest.scala#L15
https://github.com/mozilla/telemetry-batch-view/blob/4f1563d/src/test/scala/com/mozilla/telemetry/CrashAggregateViewTest.scala#L154

That adds up to 98304 pings. Depending on the exact size of each ping, we could easily get into hundreds of megabytes or even gigabytes (2k per ping = 200M, 20k per ping = 2G)

Do we really need every combination here for the test to be valid?

Comment 3

[Mauro Doglio [:mdoglio]]

I don't think we do. I would put this bug on hold for now, as bug 1357793 may make this one a WONTFIX.

Comment 4

[Firefox Bug Husbandry Bot]

Moved, per https://bugzilla.mozilla.org/show_bug.cgi?id=1453996