1527883 - Maximum element size is too restrictive for virtual scrolling

Status: NEW

(Core :: Layout: Scrolling and Overflow, defect, P5)

Description

[Andy Earnshaw]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36

Steps to reproduce:

We are using a common technique for virtual scrolling in a list (1-dimensional) and a grid (2-dimensional). The technique involves using a container element of fixed dimensions, with a child that is the size of the area you want to scroll, then the element that updates as you scroll sticky-positioned to the maximum size of the container.

Here's a rudimentary example:

https://jsfiddle.net/AndyE/2ev53nuq/56/

This works well, until you have customers with over 10 million items in a list, and those are transposed into columns on a grid.

Actual results:

Firefox tops out at 17,187,496px for the width, height, scrollWidth or scrollHeight of an element. This limits us to 429,687 rows at the 40 pixels per row given in the example. If you update the fiddle to display more than that, Firefox resets the sizer element's height to 0px (other browsers cap, I've filed a separate bug for this). If the list items are transposed into column labels, then this amount reduces drastically.

Expected results:

Naturally, our product has searching and filtering to provide more sensible navigation of large datasets. However, we still want this to work as expected if we scroll the user to a section of the grid or list based on their search results. About 64 of our customer's lists have more than 10 million items, about 470 have between 1 and 10 million and there are about 35,000 lists in total.

At the moment, we're using a custom scrollbar, but this gets too complex as you try to match the features and styling of native scrollbars.

I have heard, anecdotally whilst chasing people in my company and asking them, that we have a customer with a list of 35 million items. To virtually scroll over that many items in columns, at an average of 250px per column, we would need a scrollWidth of nearly 9 billion pixels, more than 500 times the current maximum in Firefox and almost 270 times the maximum in Chrome.

My preference is that the standards provide a better way of having virtual scrollbars than resorting to jumping through hoops like creating ridiculously large elements; I have proposed a CSSOM feature and attempted to start a discussion around web components. In the latter ticket, Domenic Denicola recommended I file tickets with the major browsers to see if I could get the limits increased instead, and this is what I am doing.

Comment 1

[Stefan [:StefanG_QA]]

Since this is an enhancement request I`m assigning it to the proper component.

Comment 2

[Andy Earnshaw]

Here's a related ticket over on the Chrome bug tracker: https://bugs.chromium.org/p/chromium/issues/detail?id=932109. So far, they don't seem keen on the idea of increasing the maximum sizes. It would be great to get some feedback here.

Comment 3

[Emilio Cobos Álvarez [:emilio]]

(In reply to Andy Earnshaw from comment #0)

Firefox tops out at 17,187,496px for the width, height, scrollWidth or scrollHeight of an element. This limits us to 429,687 rows at the 40 pixels per row given in the example. If you update the fiddle to display more than that, Firefox resets the sizer element's height to 0px (other browsers cap, I've filed a separate bug for this). If the list items are transposed into column labels, then this amount reduces drastically.

Sounds like this may just be overflowing the internal fixed-point representation of a CSS pixel.

Only way of feasibly increasing this limit would be increasing the number of bits it uses, which may or may not be desirable but it's definitely a large change...

Do you have the bug number for the clamping issue handy? Generally it's good to put related bugs in the "See also" field or such.

Comment 4

[Andy Earnshaw]

Only way of feasibly increasing this limit would be increasing the number of bits it uses, which may or may not be desirable but it's definitely a large change...

That's pretty much what Chrome said. Long term they're interested in doing this but not in the foreseeable future. This is something I expected, so I'm trying to get support for an alternative API (or proposal) specifically for enabling virtual scrolling.

Do you have the bug number for the clamping issue handy? Generally it's good to put related bugs in the "See also" field or such.

Done.

Comment 5

[Andy Earnshaw]

@AnneVK suggested I show that others are also experiencing this issue, so here's a bunch of links to open source project issue trackers where people have encountered this problem:

SlickGrid — Virtualization fails above 50k recods in IE (title mentions IE but other browsers are also discussed)
Clusterize.js — Cannot scroll past 1,342,180th line
ngx-virtual-scroller — What in case for example 1 million records

Another project, react-virtualized (probably the most popular of the ones above), uses the same workaround we are using to get around the limit, but the author created an issue to discuss the drawbacks of the workaround and potential solutions.