I evaluated performance of four documents:
using two methods:
- 1ms profiler time, and
l10n_end - l10n_trigger memory
- talos tests
From the profiler, I used the opt build, and measured
l10n_end - l10n_start and
l10n_end - l10n_trigger - the former being similarly noisy to
talos, and the latter being much cleaner. The latter is the real different, the phase where localization is applied. If you look at the profiles, almost nothing happens before than, as we don't currently
prefetch, so we can focus on the
end - trigger phase.
We need to recognize, that the profiler adds some overhead and in theory may give us different results, so it is important to cross-check with talos, but in this case, I think the results are quite consistent and Talos matches
end - start in the Profiler results, while
end - trigger is the isolated difference that represents the actual perf difference from the change.
There's also a little bit of first-run difference, so I used an average between 2nd and 3rd for the table below (stdev between them is low):
Both numbers, time and memory, go significantly down!
Unfortunately, talos tests are quite noisy, so it's really hard to pin-point the wins, but one of the wins with the patches is that the
stdev goes noticeably down, so I hope to also make the talos tests a better tool for further optimizations evaluation.
I tried to run it with ~40 reps, but stdev is continuously high enough that cutting 3ms from
browser.xhtml or even 10ms from
about:preferences is indistinguishable from noise when stdev is 15-20ms!
In result, my read from talos is that most numbers go down, in several cases quite significantly. stdev also goes down, which is great for the value of talos further :)
Here's the full compare view: https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=8771dfdc8694a91053b5e86c0a8ad9de34b68393&newProject=try&newRevision=c7f8b45423c3f228ad170c0a9b668e424f9abc96
With porfiler wins in both time and memory, and talos showing general trend down, some strong wins and much lower stdev in all tests, I'm comfortable recommending this change with the numbers as we have them right now.
Once we're closer to landing, I'll redo the talos tests to see if maybe we get more significant wins.