When running tests for the upcoming Youtube playback performance benchmark (bug 1548845) the generated gecko profiles are huge! As such the current handling of its data causes lots of problems, because it is way to slow. Here some ideas:
console.log() for the profile data before sending the data to the Python harness. This could add a hundreds of MB entry to the console, and causes a slowdown.
The Raptor web extension tries to send the full profile data via a single packet over the local / remote (android) socket connection. Imagine what could happen when you do that with >100 MB or more data.
The best way to improve the performance would be to let the Raptor web extension directly save the profile to disk, and only inform the harness of the path. Sadly this is restricted for web extensions beside storing data via local storage.
Given that the profiler is a Gecko only thing anyway, I will check if I can build an addition via WebExtensions Experiments, which would allow us to access the profiler directly. That way we would simply call