1480077 - Integer overflow in codegen for Atomics.store and assembly spew

Status: RESOLVED FIXED

(Core :: JavaScript Engine: JIT, defect)

Description

[André Bargull [:anba]]

Test case:
---
var ta = new Int32Array(new SharedArrayBuffer(4096 * Int32Array.BYTES_PER_ELEMENT));
function f() {
    for (var i = 0; i < 100; ++i) {
        try{
            Atomics.store(ta, 0x20000000, i);
        } catch {}
    }
}
for (var i = 0; i < 15; ++i) f();
---


Run with: "--ion-eager --no-threads --spectre-mitigations=off"


UBSan output:
---
/home/andre/hg/mozilla-inbound/js/src/jit/CodeGenerator.cpp:11866:54: runtime error: signed integer overflow: 536870912 * 4 cannot be represented in type 'int'
/home/andre/hg/mozilla-inbound/js/src/jit/x86-shared/BaseAssembler-x86-shared.h:2137:58: runtime error: negation of -2147483648 cannot be represented in type 'int32_t' (aka 'int'); cast to an unsigned type to negate this value to itself
---

Comment 1

[André Bargull [:anba]]

Attachment: bug1480077.patch

Simply changes the types to 'unsigned' to avoid the signed integer overflows.

The generated assembler code still contains bogus address offsets, but preceding bounds checks (and spectre index masking) should ensure that the code with the bogus offsets is never executed. At first I was considering to properly handle the offset computation using CheckedInt32, but then I saw that more code was simply using unsigned ints for similar address computations [1].

[1] https://searchfox.org/mozilla-central/search?q=symbol:_ZN2js3jitL7ToInt32EPKNS0_11LAllocationE&redirect=false

Comment 2

[Lars T Hansen [:lth]]

Comment on attachment 8996694 [details] [diff] [review]
bug1480077.patch

Review of attachment 8996694 [details] [diff] [review]:
-----------------------------------------------------------------

Seems reasonable.

Comment 3

[André Bargull [:anba]]

Attachment: bug1480077.patch

Updated to adjust review name and to change a macro to please MSVC. (*)

(*) MSVC didn't like |-unsigned(x)|, complaining about "warning C4146: unary minus operator applied to unsigned type, result still unsigned".

Comment 4

[André Bargull [:anba]]

Try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5dafa3f50d04bf9bc793cab64e8caf4653b907dc

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/a6b2aa1c9657
Avoid signed integer overflow in Atomics.store and when printing assembler code. r=lth

Comment 6

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/a6b2aa1c9657