Make ArrayBuffer's handling of kinds and data-ownership sane and simple and understandable
Categories
(Core :: JavaScript Engine, enhancement)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox67 | --- | fixed |
People
(Reporter: Waldo, Assigned: Waldo)
References
Details
Attachments
(39 files)
The complexity has sprawled. I fear to review or touch the code because I can't understand it enough to safely touch it. Bug 1502562 was the last straw, where like a five-line patch I felt safer/more confident not taking and leaking, than taking and not leaking but maybe terrible things happen too?
So, have a gonzo patch stack of mostly a zillion smallish transformation, only a few of which actually change how the world functions, and in the end we have an understandable system.
sfink has volunteered as tribute. Or been volunteered; I forget which.
| Assignee | ||
Comment 1•6 years ago
|
||
| Assignee | ||
Comment 2•6 years ago
|
||
| Assignee | ||
Comment 3•6 years ago
|
||
| Assignee | ||
Comment 4•6 years ago
|
||
| Assignee | ||
Comment 5•6 years ago
|
||
| Assignee | ||
Comment 6•6 years ago
|
||
| Assignee | ||
Comment 7•6 years ago
|
||
| Assignee | ||
Comment 8•6 years ago
|
||
| Assignee | ||
Comment 9•6 years ago
|
||
| Assignee | ||
Comment 10•6 years ago
|
||
| Assignee | ||
Comment 11•6 years ago
|
||
| Assignee | ||
Comment 12•6 years ago
|
||
| Assignee | ||
Comment 13•6 years ago
|
||
| Assignee | ||
Comment 14•6 years ago
|
||
| Assignee | ||
Comment 15•6 years ago
|
||
| Assignee | ||
Comment 16•6 years ago
|
||
| Assignee | ||
Comment 17•6 years ago
|
||
| Assignee | ||
Comment 18•6 years ago
|
||
| Assignee | ||
Comment 19•6 years ago
|
||
| Assignee | ||
Comment 20•6 years ago
|
||
| Assignee | ||
Comment 21•6 years ago
|
||
| Assignee | ||
Comment 22•6 years ago
|
||
| Assignee | ||
Comment 23•6 years ago
|
||
| Assignee | ||
Comment 24•6 years ago
|
||
| Assignee | ||
Comment 25•6 years ago
|
||
| Assignee | ||
Comment 26•6 years ago
|
||
| Assignee | ||
Comment 27•6 years ago
|
||
| Assignee | ||
Comment 28•6 years ago
|
||
| Assignee | ||
Comment 29•6 years ago
|
||
| Assignee | ||
Comment 30•6 years ago
|
||
| Assignee | ||
Comment 31•6 years ago
|
||
| Assignee | ||
Comment 32•6 years ago
|
||
| Assignee | ||
Comment 33•6 years ago
|
||
| Assignee | ||
Comment 35•6 years ago
|
||
| Assignee | ||
Comment 36•6 years ago
|
||
| Assignee | ||
Comment 37•6 years ago
|
||
| Assignee | ||
Comment 38•6 years ago
|
||
And that's it! ("that") Hopefully I kept everything small enough this should be mostly simple to run through begin to end.
I can also upload ABO.{cpp,h} and StructuredClone.cpp if the full final states of those files ends up helpful for understanding where things end up, if you want.
Comment 39•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Comment 40•6 years ago
|
||
| Assignee | ||
Comment 41•6 years ago
|
||
(In reply to Steve Fink [:sfink] [:s:] from comment #40)
- using BufferContents = ArrayBufferObject::BufferContents;
using ArrayBufferObject::BufferContents;doesn't work?
That's the syntax for accessing inherited members (using declarations), not for declaring type aliases. I'm not sure why C++ doesn't let you use both when there's not ambiguity or something.
Ugh. This stuff all sucks. Hopefully this patch stack will make it tolerable?
Maybe you think it be like it is, and it do.
- /**
* User-owned memory. The associated buffer must be manually detached* before the user-owned memory becomes invalid.On first read, this sounded very wrong -- it sounded to me like you should
manually detach in order to correctly invalidate the memory, which doesn't
make sense and confused me. I just automatically read it as "this is the
correct series of events: (1) detach, (2) the memory becomes invalid."Uh... "The owner must not allow the memory to become invalid without
manually detaching it first."? "The owner must keep the memory valid until
the associated buffer is detached."? I don't know.
Some patch somewhere, maybe this bug for all I remember, adds docs by JS_NewArrayBufferWithExternalContents (or its renaming, can't remember where that is either) that says what you have to do when. IMO.
| Assignee | ||
Comment 42•6 years ago
|
||
Try-run is surprisingly clean, only four failures in the whole of it without even categorizing anything or anything.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e44562b5cdadea69fec19c362812ad5258bcfd37
Comment 43•6 years ago
|
||
| Assignee | ||
Updated•6 years ago
|
| Assignee | ||
Comment 44•6 years ago
|
||
Updated•6 years ago
|
Comment 45•6 years ago
|
||
Comment 46•6 years ago
|
||
| bugherder | ||
Comment 47•6 years ago
|
||
Comment 48•6 years ago
|
||
Updated•6 years ago
|
Comment 49•6 years ago
|
||
Comment 50•6 years ago
|
||
Comment 51•6 years ago
|
||
Updated•6 years ago
|
Comment 52•6 years ago
|
||
Comment 53•6 years ago
|
||
Comment 54•6 years ago
|
||
Updated•6 years ago
|
Comment 55•6 years ago
|
||
Updated•6 years ago
|
Comment 56•6 years ago
|
||
Comment 57•6 years ago
|
||
Backed out for spidermonkey bustages on /testAtomicOperations.cpp
backout: https://hg.mozilla.org/integration/mozilla-inbound/rev/eed1098d0d6c9e3af5b02154295e452c6c21bb48
failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229805289&repo=mozilla-inbound&lineNumber=2861
[task 2019-02-22T03:05:14.469Z] In file included from /builds/worker/workspace/build/src/obj-spider/js/src/jsapi-tests/Unified_cpp_js_src_jsapi-tests1.cpp:2:0:
[task 2019-02-22T03:05:14.469Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp: In member function 'virtual bool cls_testAtomicOperationsU8Clamped::run(JS::HandleObject)':
[task 2019-02-22T03:05:14.469Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:294:9: error: 'uint8_clamped' does not name a type
[task 2019-02-22T03:05:14.469Z] const uint8_clamped A(0xab);
[task 2019-02-22T03:05:14.469Z] ^~~~~~~~~~~~~
[task 2019-02-22T03:05:14.469Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:295:9: error: 'uint8_clamped' does not name a type
[task 2019-02-22T03:05:14.470Z] const uint8_clamped B(0x37);
[task 2019-02-22T03:05:14.470Z] ^~~~~~~~~~~~~
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:296:24: error: 'uint8_clamped' was not declared in this scope
[task 2019-02-22T03:05:14.470Z] ATOMIC_CLAMPED_TESTS(uint8_clamped, A, B);
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:283:3: note: in definition of macro 'ATOMIC_CLAMPED_TESTS'
[task 2019-02-22T03:05:14.470Z] T* q = (T*)hidePointerValue((void*)atomicMem);
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:283:6: error: 'q' was not declared in this scope
[task 2019-02-22T03:05:14.470Z] T* q = (T*)hidePointerValue((void*)atomicMem);
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:296:3: note: in expansion of macro 'ATOMIC_CLAMPED_TESTS'
[task 2019-02-22T03:05:14.470Z] ATOMIC_CLAMPED_TESTS(uint8_clamped, A, B);
[task 2019-02-22T03:05:14.470Z] ^~~~~~~~~~~~~~~~~~~~
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:283:13: error: expected primary-expression before ')' token
[task 2019-02-22T03:05:14.470Z] T* q = (T*)hidePointerValue((void*)atomicMem);
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:296:3: note: in expansion of macro 'ATOMIC_CLAMPED_TESTS'
[task 2019-02-22T03:05:14.470Z] ATOMIC_CLAMPED_TESTS(uint8_clamped, A, B);
[task 2019-02-22T03:05:14.470Z] ^~~~~~~~~~~~~~~~~~~~
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:296:39: error: 'A' was not declared in this scope
[task 2019-02-22T03:05:14.470Z] ATOMIC_CLAMPED_TESTS(uint8_clamped, A, B);
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:284:8: note: in definition of macro 'ATOMIC_CLAMPED_TESTS'
[task 2019-02-22T03:05:14.470Z] q = A;
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:285:15: error: template argument 1 is invalid
[task 2019-02-22T03:05:14.470Z] SharedMem<T> p =
[task 2019-02-22T03:05:14.470Z] ^
[task 2019-02-22T03:05:14.470Z] /builds/worker/workspace/build/src/js/src/jsapi-tests/testAtomicOperations.cpp:296:3: note: in expansion of macro 'ATOMIC_CLAMPED_TESTS'
[task 2019-02-22T03:05:14.470Z] ATOMIC_CLAMPED_TESTS(uint8_clamped, A, B);
[task 2019-02-22T03:05:14.470Z] ^~~~~~~~~~~~~~~~~~~~
Comment 58•6 years ago
|
||
| Assignee | ||
Comment 59•6 years ago
|
||
Comment 60•6 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/efd2d5e7ec57
https://hg.mozilla.org/mozilla-central/rev/c0aee08543ad
https://hg.mozilla.org/mozilla-central/rev/9e5e10661a78
https://hg.mozilla.org/mozilla-central/rev/76600605e1e3
https://hg.mozilla.org/mozilla-central/rev/5a304d4b53c0
https://hg.mozilla.org/mozilla-central/rev/765bc7a86f72
https://hg.mozilla.org/mozilla-central/rev/92fc3d9b6699
https://hg.mozilla.org/mozilla-central/rev/51d373ab477b
https://hg.mozilla.org/mozilla-central/rev/4771a589408d
https://hg.mozilla.org/mozilla-central/rev/b2c09226e55b
https://hg.mozilla.org/mozilla-central/rev/76e407233988
https://hg.mozilla.org/mozilla-central/rev/dcdddf46e820
https://hg.mozilla.org/mozilla-central/rev/1cead3a873e2
https://hg.mozilla.org/mozilla-central/rev/3556d94cbc81
https://hg.mozilla.org/mozilla-central/rev/5b7a07b449ba
| Assignee | ||
Updated•6 years ago
|
Comment 61•6 years ago
|
||
Comment 62•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Comment 63•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Comment 64•6 years ago
|
||
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Updated•6 years ago
|
Comment 65•6 years ago
|
||
Comment 66•6 years ago
|
||
Comment 67•6 years ago
|
||
Updated•6 years ago
|
Comment 68•6 years ago
|
||
| Assignee | ||
Comment 69•6 years ago
|
||
(In reply to Steve Fink [:sfink] [:s:] from comment #63)
::: js/src/vm/StructuredClone.cpp
@@ +1875,2 @@if (!bufContents) { return false; // out of memoryHm... there seems to be an OOM reporting problem here, in some earlier patch
that I missed.This condition is now OOM or WASM, though the wasm case will never happen
because
https://searchfox.org/mozilla-central/rev/
dbddac86aadf1d4871fb350bbe66db43728a9f81/js/src/vm/StructuredClone.cpp#1838I see
https://searchfox.org/mozilla-central/rev/
dbddac86aadf1d4871fb350bbe66db43728a9f81/js/src/vm/StructuredClone.cpp#1838
that reports OOM, but then all the other paths do not. Given
https://searchfox.org/mozilla-central/rev/
dbddac86aadf1d4871fb350bbe66db43728a9f81/js/src/vm/StructuredClone.cpp#1903
it really seems like everything should report OOM.Perhaps this should be
if (!bufContents) { MOZ_ASSERT(!arrayBuffer->isWasm()); ReportOutOfMemory(cx); return false; }at least for now?
As of this patch and the prior ones, you have
/* static */ ArrayBufferObject::BufferContents
ArrayBufferObject::extractStructuredCloneContents(
JSContext* cx, Handle<ArrayBufferObject*> buffer) {
CheckStealPreconditions(buffer, cx);
BufferContents contents = buffer->contents();
switch (contents.kind()) {
case INLINE_DATA:
case NO_DATA:
case USER_OWNED: {
uint8_t* copiedData = NewCopiedBufferContents(cx, buffer);
if (!copiedData) {
return BufferContents::createFailed();
}
ArrayBufferObject::detach(cx, buffer, BufferContents::createNoData());
return BufferContents::createMalloced(copiedData);
}
case MALLOCED:
case MAPPED: {
MOZ_ASSERT(contents);
// Overwrite the old data pointer *without* releasing old data.
BufferContents newContents = BufferContents::createNoData();
buffer->setDataPointer(newContents, OwnsData);
// Detach |buffer| now that doing so won't release |oldContents|.
ArrayBufferObject::detach(cx, buffer, newContents);
return contents;
}
case WASM:
JS_ReportErrorNumberASCII(cx, GetErrorMessage, nullptr,
JSMSG_WASM_NO_TRANSFER);
return BufferContents::createFailed();
case EXTERNAL:
MOZ_ASSERT_UNREACHABLE(
"external ArrayBuffer shouldn't have passed the "
"structured-clone preflighting");
break;
case BAD1:
MOZ_ASSERT_UNREACHABLE("bad kind when stealing malloc'd data");
break;
}
MOZ_ASSERT_UNREACHABLE("garbage kind computed");
return BufferContents::createFailed();
}
The external/bad1/fallthrough cases can be silent halt to script execution, that's fine and perhaps even good. WASM will obviously set a pending exception. MALLOCED/MAPPED always return non-null contents. INLINE/NO/USER if NewCopiedBufferContents fails, it reported OOM -- I changed this earlier in this bug -- so we're good there. And then the copiedData returned is non-null.
So I don't think there's an OOM reporting problem here. I think you're comparing against current code without remembering everything all these other patches here have done?
Comment 70•6 years ago
|
||
| Assignee | ||
Comment 71•6 years ago
|
||
(In reply to Steve Fink [:sfink] [:s:] from comment #67)
That was an awesome refactoring series. Thank you for breaking it down to
that level, it really helped.
Didn't much have a choice, as I wouldn't have been smart enough to do it any other way. :-) Gordian knot, you have to pull a little bit at a time to get the whole thing untangled given the unholy kind/ownership pairing.
Comment 72•6 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/ac3ad3942d67
https://hg.mozilla.org/mozilla-central/rev/d8a4ce77f2f1
https://hg.mozilla.org/mozilla-central/rev/f487c864d2ab
https://hg.mozilla.org/mozilla-central/rev/715e9b139ebb
https://hg.mozilla.org/mozilla-central/rev/a9638eeea757
https://hg.mozilla.org/mozilla-central/rev/63536a044a29
https://hg.mozilla.org/mozilla-central/rev/5cb592de5e03
https://hg.mozilla.org/mozilla-central/rev/a7a39ff06158
https://hg.mozilla.org/mozilla-central/rev/713088adfe5f
https://hg.mozilla.org/mozilla-central/rev/870a55710969
https://hg.mozilla.org/mozilla-central/rev/f4101f442782
https://hg.mozilla.org/mozilla-central/rev/a06864bc8352
https://hg.mozilla.org/mozilla-central/rev/ef4c27821811
https://hg.mozilla.org/mozilla-central/rev/650bd5a18809
https://hg.mozilla.org/mozilla-central/rev/694fe0c43793
https://hg.mozilla.org/mozilla-central/rev/973c3800d5d6
https://hg.mozilla.org/mozilla-central/rev/ab967077f8a3
https://hg.mozilla.org/mozilla-central/rev/9103748036d1
https://hg.mozilla.org/mozilla-central/rev/bb879a6a95f8
Description
•