Luke and I have had a conversation with Jacob Bramley from ARM and the conclusion of this discussion is that, at the moment, we are (almost) fine. I'll try to summarize the discussion briefly.
The setup is that we have a jump table per module, with one slot per function. On function entry, baseline-compiled code loads the slot corresponding to the function and jumps to that address. The initial value of the slot for a function is the address of the instruction following the jump.
When we tier the module, we background-compile the code and then make the machine code available:
- get a writable page (which may have previously been executable code that was freed)
- write the new machine code onto the page
- mprotect the page to executable
- racily store pointers to the new code in the jump table
Thus baseline functions will jump to their Ion versions on entry (but baseline functions stuck in loops, say, will never reach their Ion versions).
Step 3 happens in some background compiler thread and can be expensive and slow, but once Step 4 does the store, the code immediately becomes available to N other executing threads.
Jacob says he thinks this is safe (as in, no core will be seeing old, stale instructions), based on some assumptions:
(a) Step 1 must not re-use the page(s) that held the previous version of the code.
(b) Executing cores observe the pointer updates (4) after the new code is written (2).
(c) Pointer loads and stores (4) are (single-copy) atomic.
(d) At the beginning of step 2, no core is executing anything from the writeable page(s).
All of these hold in Firefox currently, except that we probably want to execute a DSB before step 3 so that we are sure that mprotect does not risk making a page executable before all the writes have reached memory and have been observed by all cores; there's a theoretical risk of this if mprotect only operates on the local core.
What makes it safe is that while cores can read ahead and thus technically see old instructions after new instructions become available, the branch through the branch table will have been mispredicted and hence those old instructions must be discarded.
In summary: no security issue in current Firefox but we'll probably want to get that DSB in there just for the sake of being safe; but if we do more fine-grained things with patching in the future we may need to worry about getting the ISB in there.