1248405 - CanOptimizeForDenseStorage should distinguish OOM and not-optimizable cases.

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Tooru Fujisawa [:arai] (away until 08-17)]

>     ObjectGroup* arrGroup = arr->getGroup(cx);
>     if (MOZ_UNLIKELY(!arrGroup || arrGroup->hasAllFlags(OBJECT_FLAG_ITERATED)))
>         return false;

Here, returning false means 2 things, OOM and not-optimizable.
We should have out parameter for optimizable/not-optimizable.

Comment 1

[Tooru Fujisawa [:arai] (away until 08-17)]

Attachment: Make CanOptimizeForDenseStorage fallible and handle JSObject::getGroup OOM.

added optimizable out parameter to CanOptimizeForDenseStorage, and fixed caller accordingly.

Comment 2

[Jan de Mooij [:jandem]]

I'm not sure about this patch - it makes the call sites more verbose and it should be fine to use the slow path on OOM. What if we do something like:

if (!arrGroup) {
    cx->recoverFromOutOfMemory();
    return false;
}

Comment 3

[Tooru Fujisawa [:arai] (away until 08-17)]

Attachment: Recover from JSObject::getGroup OOM in CanOptimizeForDenseStorage.

Yeah, I agree this also works :)

Comment 4

[Jan de Mooij [:jandem]]

Comment on attachment 8719791 [details] [diff] [review]
Recover from JSObject::getGroup OOM in CanOptimizeForDenseStorage.

Review of attachment 8719791 [details] [diff] [review]:
-----------------------------------------------------------------

Thank you.

Comment 5

[Tooru Fujisawa [:arai] (away until 08-17)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/bda2d0f0622c578748d5eeadb32942575c2f83b5
Bug 1248405 - Recover from JSObject::getGroup OOM in CanOptimizeForDenseStorage. r=jandem

Comment 6

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/bda2d0f0622c