1686445 - Make structured cloning work with large array buffers

Status: RESOLVED FIXED

(Core :: JavaScript Engine, task, P1)

Description

[Jan de Mooij [:jandem]]

This will need format changes for ArrayBuffer, TypedArray, DataView objects because it stores (some) length values as 32-bit integers. We still need to support reading the current format, so we also have to add tests for that.

I wrote some patches for this a few months ago that I just have to clean up.

Comment 1

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 1 - Add structured clone tests for reading current array buffer format. r?sfink!

This ensures we can still read this after we add the new V3 types.

Comment 2

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 2 - Add structured clone support for large ArrayBuffers. r?sfink!

Renames SCTAG_ARRAY_BUFFER_OBJECT to SCTAG_ARRAY_BUFFER_OBJECT_V2 and adds a new
SCTAG_ARRAY_BUFFER_OBJECT that encodes the length as 64-bit value.

Depends on D101734

Comment 3

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 3 - Add structured clone support for large TypedArrays. r?sfink!

Instead of storing the length in the 32-bit data and the array type separately as
64-bit value, the new version just switches those.

Depends on D101735

Comment 4

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 4 - Add structured clone support for large DataViews. r?sfink!

Depends on D101736

Comment 5

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 5 - Add structured clone support for large SharedArrayBuffers. r?sfink!,lth!

There are no backwards-compatibility concerns because SABs are only cloned within
the same process (checked in writeSharedArrayBuffer).

Depends on D101737

Comment 6

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 6 - Add structured clone support for transfering large ArrayBuffers. r?sfink!

The length is encoded as 64-bit value (extraData) and used within the same process.

Depends on D101738

Comment 7

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 7 - Change some APIs to take int64_t/size_t instead of int32_t/uint32_t. r?sfink!

This prevents truncation in the structured clone code.

Depends on D101739

Comment 8

[Jan de Mooij [:jandem]]

Attachment: Bug 1686445 part 8 - Add tests for structured cloning large buffers. r?sfink!

Depends on D101740

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/eae09146c9f9
part 1 - Add structured clone tests for reading current array buffer format. r=sfink
https://hg.mozilla.org/integration/autoland/rev/fae906026e24
part 2 - Add structured clone support for large ArrayBuffers. r=sfink
https://hg.mozilla.org/integration/autoland/rev/ad251c4f6d76
part 3 - Add structured clone support for large TypedArrays. r=sfink
https://hg.mozilla.org/integration/autoland/rev/2ca9f9036962
part 4 - Add structured clone support for large DataViews. r=sfink
https://hg.mozilla.org/integration/autoland/rev/7f17f8fb1135
part 5 - Add structured clone support for large SharedArrayBuffers. r=sfink,lth
https://hg.mozilla.org/integration/autoland/rev/f7ac99e85da7
part 6 - Add structured clone support for transfering large ArrayBuffers. r=sfink
https://hg.mozilla.org/integration/autoland/rev/ea90adb7bf61
part 7 - Change some APIs to take int64_t/size_t instead of int32_t/uint32_t. r=sfink
https://hg.mozilla.org/integration/autoland/rev/d5154885d8d2
part 8 - Add tests for structured cloning large buffers. r=sfink

Comment 10

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/eae09146c9f9
https://hg.mozilla.org/mozilla-central/rev/fae906026e24
https://hg.mozilla.org/mozilla-central/rev/ad251c4f6d76
https://hg.mozilla.org/mozilla-central/rev/2ca9f9036962
https://hg.mozilla.org/mozilla-central/rev/7f17f8fb1135
https://hg.mozilla.org/mozilla-central/rev/f7ac99e85da7
https://hg.mozilla.org/mozilla-central/rev/ea90adb7bf61
https://hg.mozilla.org/mozilla-central/rev/d5154885d8d2