The graph shows the current heap limits compared with the optimal heap limit approach described in the paper. (Optimal heap limit graph is shown for allocation rate / collection rate = 0.03).
Currently, we choose either the low frequency or high frequency limit based on how often GCs are occurring, with the high frequency version if GCs happen more than once a second. There is a hard cutover between the two.
The high frequency limit (which ends up being used for GC heavy workload) uses a variable multiplicative factor based on the size of the heap. This is linearly interpolated between two values for mid-sized heaps and this results in the 'bump' in the graph.
The hard cutover between the two regimes and the bump in the latter one mean that similar workloads (or even the same workload) can end up with very different GC behaviour. I believe this is one reason we see noisy or bimodal behaviour in our tests.
The optimal heap limit calculation should not have this behaviour. It takes into account allocation rate directly so it doesn't need the cutover to boost heap sizes for GC heavy workloads and it changes smoothly based on heap size.