1805285 - Speedometer 2 score improves by ~5% on some machines when restricting core count

Status: NEW

(Core :: Performance Engineering, defect, P3)

Description

[Alex Thayer [:alexical] (she/her)]

This came up due to a conversation with tcampbell, where he noted that he was seeing performance improvements on Speedometer on his machine with a high core count by restricting the number of active cores.

I decided to see if I could repro this on my machine. I set the process affinity mask in Process Explorer for the content process running Speedometer to a few different permutations. What I observed is restricting it to 4 physical cores gave me the highest speedometer score over 5 or so runs. I am on a AMD Ryzen Threadripper with 32 physical cores.

I don't want to go too deep down the rabbit hole at this stage but I strongly suspect that what is happening here is we are ending up allocating memory in VirtualAlloc from a different NUMA domain on, say, an Ion compilation thread, and then using that memory from the main thread. The result being that our accesses to main memory are costlier than they would be if we were operating only on one NUMA domain, which restricting the process's affinity mask does for us. To corroborate this, I set the affinity mask to use 8 virtual cores in different NUMA domains, and saw the worst performance yet.

This might not be relevant for typical users today, but it plausibly affects our CI numbers (?), it could affect our numbers from tech journalists benchmarking on high-end hardware, and I think more and more consumer hardware is going to be NUMA in the future.

Comment 1

[Ted Campbell [:tcampbell]]

Thanks for validating this! I was seeing a similar peak of 4 cores uses Ubuntu on a 28-core Xeon machine which is also a NUMA setup.

Comment 2

[Mayank Bansal]

Could this be because restricting the core count leads to the used cores ramping at higher frequency (within the same power budget)

Comment 3

[Bas Schouten (:bas.schouten)]

Fwiw, at least modern intel consumer intel chips like Alder Lake are not implemented as NUMA. My Ryzen 5950X is similarly in a single NUMA domain afaict. I don't think NUMA is going to become common in consumer hardware in the near future for exactly the reasons you found :-). Similarly the chip we use in CI (https://ark.intel.com/content/www/us/en/ark/products/93741/intel-xeon-processor-e3-1585l-v5-8m-cache-3-00-ghz.html) does not support NUMA. So I doubt this is relevant for most. You may also very well find similar differences in Chrome? It'd be interesting if you didn't!

Comment 4

[Alex Thayer [:alexical] (she/her)]

(In reply to Mayank Bansal from comment #2)

Could this be because restricting the core count leads to the used cores ramping at higher frequency (within the same power budget)

After turning off dynamic frequency scaling, you appear to be correct. The effect has disappeared. I'm not much of a hardware person, so forgive me, but given that the effect never showed up when I set the affinity mask to only cores from different dies, I'm speculating that the feature is constrained to scale frequency on a per-die basis? In which case we could potentially see this kind of win on any processor with multiple dies?

Comment 5

[Alex Thayer [:alexical] (she/her)]

I'm setting this as a P3 right now, because it's a lot of Speedometer score to leave on the table, but I think all signs point to it being of limited applicability to real world scenarios. However I still think it's something that could be worth looking at.

To answer Bas's question about whether this same effect shows up when benchmarking Chrome: yes, it does. So Chrome hasn't optimized it yet. I do however suspect that it would show up on your Ryzen 5950X, Bas, and it may be worth giving it a go.

I would expect that, today, this effect will show up for anyone on a processor with 16 or more physical cores, running any mostly-single-threaded JS-heavy extended workload. The most common occurrence of this workload is going to be benchmarks.

Also, if anyone is on an Apple M1 or M2, I would be curious to see if this effect shows up for you.

Comment 6

[BugBot [:suhaib / :marco]]

The severity field is not set for this bug.
:fdoty, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 7

[Paul Bone [:pbone]]

I also saw this on an i9-7940x (14 cores). When I fixed the frequency to 3.1Ghz the effect disappeared. (Without fixing the frequency when running single core this could boost up to 4.3Ghz).

Comment 8

[Patricia Lawless]

Moving to S3 because as Doug states in comment 5: "all signs point to it being of limited applicability to real world scenarios"