Open Bug 1153865 Opened 5 years ago Updated 5 years ago
Use a segmented hash table in the CCGraph
(In reply to David Major [:dmajor] from bug 1144649 comment #14) > Allocations over 1 meg really should be expected to fail, and the larger the > size, the more we should care. Large contiguous regions are hard to find on > Windows. The users who fail at the 4M and 8M sizes likely could have kept > going for a while before reaching hopeless OOM|small territory. > > So I think this is worth doing something about. Either by gracefully > handling failure or by playing tricks to keep the individual allocations > smaller. (In reply to Nathan Froyd [:froydnj] [:nfroyd] from bug 1144649 comment #16) > Maybe we could make (template?) a variant of PLDHashTable that uses > SegmentedVector or similar underneath, so growing the hashtable would make > lots of smaller allocations rather than one large one?
Summary: Use a segmented data structure in the CCGraph → Use a segmented hash table in the CCGraph
It would be cool if all of our hash tables could switch to the segmented strategy if they get too big...
Well, if we can come up with a way to do it without any additional performance cost in the non-segmented case.
Whiteboard: [MemShrink] → [MemShrink:P2]
Of course with segmented storage you pay a cache locality price, so you will probably want to measure CC times if you end up doing anything here.
Yeah. When I last measured it, hash table lookups were like 20% of the CC time. I think it is less of an issue now that CC is incremental, but it is something we'd want to check, for sure.
You need to log in before you can comment on or make changes to this bug.