1355725 - Attempting to store 100*128MB of typed array data to IndexedDB steals the CPU for several minutes

Status: RESOLVED INCOMPLETE

(Core :: Storage: IndexedDB, defect, P5)

Description

[Jukka Jylänki]

Attachment: testidbquotaexceedederror.html

STR:

1. Run the attached page, open web console and OS X Activity Monitor and wait. The page attempts to store 100 Uint8Arrays to IndexedDB, each of size 128MB.
2. Slow script dialog will pop up, don't click on anything but keep waiting. Observe that CPU utilization varies between 100%-300% in Activity Monitor.
3. Eventually an AbortError message will show up on the page console, keep waiting.
4. Several minutes later a spam of many more AbortErrors will show up in the page console. CPU utilization is still at > 100%
5. A bit more later, a large amount of prints

> Successfully stored key data0
> putRequest.onsuccess
> ...
> Successfully stored key data99
> putRequest.onsuccess

appear on the page.

Expected:

a. Since IDB operations are asynchronous, they should not pop up the slow script dialog.

b. Benchmarking disk performance on the test system (MacBook Pro 13" 2016 model) with the following C application:

> #include <stdlib.h>
> #include <stdio.h>
> #include <time.h>
> 
> int main()
> {
> 	FILE *handle = fopen("test.dat", "w");
> 	char *data = new char[128*1024*1024];
> 	long t0 = time(NULL);
> 	for(int i = 0; i < 100; ++i)
> 	{
> 		printf("Writing %d\n", i);
> 		fwrite(data, 128*1024*1024, 1, handle);
> 	}
> 	fclose(handle);
> 	long t1 = time(NULL);
> 	printf("Total took %ld\n", t1-t0);
> }

prints out 18 seconds, which comes out as file write rate of about 711MB/sec. It would be good to get closer to this rate.

c. Presumably it is not currently allowed for a page to store 100*128MB=12.5GB of data to IndexedDB? I was writing the test page with the expectation of getting back QuotaExceededErrors, but that does not seem to be the case? How is the available quota computed?

d. The write operations seem to have failed since most of them raised an AbortError. However, the putRequest.onsuccess() handler is called for each operation, and the page is led to think that all of the writes would have succeeded.

e. The AbortError that is raised is directly printed to page console and did not go through the error reporting machinery (putRequest.onerror()) that IndexedDB provides. The failures should have called putRequest.onerror() to report the error?

Comment 1

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Hi Bevis, could you clarify the expected behaviour here? And how is the relation b/w AbortError and putRequest.onsuccess()? The current behaviour comment 0 reported looks so confusing...

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Jukka Jylänki from comment #0)
> a. Since IDB operations are asynchronous, they should not pop up the slow
> script dialog.

The structured clone operate is necessarily synchronous.  Uint8Arrays are stored inline and need to be serialized to the byte-stream.  Large blobs are serialized by reference and their contents sent asynchronously, so if you want to remove that overhead, wrap the arrays in separate Blobs and send those instead.  (If you use the same Blob, IndexedDB will de-duplicate it.)  WebAssembly modules are handled similarly, although not currently de-duplicated.

There was some discussion about this from Nolan Lawson's blog post at https://nolanlawson.com/2015/09/29/indexeddb-websql-localstorage-what-blocks-the-dom/ a while back, see bug 1208840 for investigation and discussion.  The testing code can still be found at https://github.com/nolanlawson/database-comparison.

> c. Presumably it is not currently allowed for a page to store
> 100*128MB=12.5GB of data to IndexedDB? I was writing the test page with the
> expectation of getting back QuotaExceededErrors, but that does not seem to
> be the case? How is the available quota computed?

https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Browser_storage_limits_and_eviction_criteria#Storage_limits

Do you have a lot of free disk space on whatever disk you are testing on?

> d. The write operations seem to have failed since most of them raised an
> AbortError. However, the putRequest.onsuccess() handler is called for each
> operation, and the page is led to think that all of the writes would have
> succeeded.
>
> e. The AbortError that is raised is directly printed to page console and did
> not go through the error reporting machinery (putRequest.onerror()) that
> IndexedDB provides. The failures should have called putRequest.onerror() to
> report the error?

You'll want to listen on the transaction for errors too, not just the request.

IndexedDB provides MOZ_LOG output that can aid in understanding when and where the errors are being generated.  Running with MOZ_LOG=IndexedDB:5 should result in debug output where logging is generated for each request both in the originating document/worker, as well as when the request is serviced in the parent process's PBackground.

Comment 3

[Jan Varga [:janv]]

Is there a reason this can't be executed one by one instead of issuing 100 requests at once ?

Comment 4

[Jukka Jylänki]

(In reply to Jan Varga [:janv] from comment #3)
> Is there a reason this can't be executed one by one instead of issuing 100
> requests at once ?

The above is not production code, the intent is to stress the browser to understand limits of API behavior.

(In reply to Andrew Sutherland [:asuth] from comment #2)
> The structured clone operate is necessarily synchronous.  Uint8Arrays are
> stored inline and need to be serialized to the byte-stream.  Large blobs are
> serialized by reference and their contents sent asynchronously, so if you
> want to remove that overhead, wrap the arrays in separate Blobs and send
> those instead.  (If you use the same Blob, IndexedDB will de-duplicate it.) 
> WebAssembly modules are handled similarly, although not currently
> de-duplicated.

That makes sense, did not realize there was a heavy synchronous part to this. Certainly can prefer to use Blobs instead.

> https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/
> Browser_storage_limits_and_eviction_criteria#Storage_limits
> 
> Do you have a lot of free disk space on whatever disk you are testing on?

Yeah, had about 700GB available. Thanks for the link!

> You'll want to listen on the transaction for errors too, not just the
> request.

Tested adding error handling to the transaction as well, though I am not seeing that error handler get called, and the success handler does still get called.

Comment 5

[Bevis Tseng[:bevis][:btseng](Exited)]

(In reply to Jukka Jylänki from comment #0)
> e. The AbortError that is raised is directly printed to page console and did
> not go through the error reporting machinery (putRequest.onerror()) that
> IndexedDB provides. The failures should have called putRequest.onerror() to
> report the error?

I've no luck to reproduce this symptom on my Mac with the same test file.

However, in general, listen to transaction.abort() is the recommended way to see if the entire transaction is either committed or aborted instead of checking each IDBRequest.onerror because it is reasonable to have multiple IDBRequest per transaction.

Once any onerror fired from one of these IDBRequests, the error event will be sent through this propagation path in indexedDB:
IDBRequest -> IDBTransaction -> IDBDatabase -> ScriptGlobal (window, worker) => window (if ScriptGlobal is a worker).
If event.preventDefault() during this propagation, the default action of onerror is to abort current transaction.
(IDBTransaction.onabort() will be fired eventually.)

For large objects to be added into IDBObjectStore, as :asuth has suggested in comment 2, Blob is the right choice to prevent the serialization/de-serialization of these object synchronously.

Comment 6

[Bevis Tseng[:bevis][:btseng](Exited)]

(In reply to Bevis Tseng[:bevistseng][:btseng] from comment #5)
> If event.preventDefault() during this propagation, the default action of
                           ^ is not invoked 
> onerror is to abort current transaction.
> (IDBTransaction.onabort() will be fired eventually after transaction is aborted.)

Comment 7

[Jens Stutte [:jstutte]]

Jukka, with the given explanations, would you still see a defect here and if yes, can you be more specific about it? Thank you!

Comment 8

[Jens Stutte [:jstutte]]

Closing for a lack of response.