627475 - JM: Clean up OOM handling

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

There are still a few places in the methodjit where OOM handling is a little off.

Comment 1

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Attachment: patch

This fixes two problems. One is that the execPool->alloc(totalSize) allocation wasn't being checked for OOM. The other is that sometimes we weren't calling js_ReportOutOfMemory for OOMs.

For the second fix, I changed the interface for reporting OOMs a little bit. Most OOM errors set one of the following flags: (oomInVector || masm.oom() || stubcc.masm.oom()). In these cases, the compiler returns as quickly as possible to performCompilation, which is responsible for doing ReportOutOfMemory. In all other cases, ReportOutOfMemory should be done as soon as the problem is discovered.

Comment 2

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

http://hg.mozilla.org/tracemonkey/rev/4031baed59f7

Comment 3

[Bill McCloskey [inactive unless it's an emergency] (:billm)]

Small fix:
http://hg.mozilla.org/tracemonkey/rev/4f9f2c9f1e64

Comment 4

[Chris Leary [:cdleary] (not checking bugmail)]

cdleary-bot mozilla-central merge info:
http://hg.mozilla.org/mozilla-central/rev/4031baed59f7
http://hg.mozilla.org/mozilla-central/rev/4f9f2c9f1e64