1324202 - UB in js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14

Status: RESOLVED INCOMPLETE

(Core :: JavaScript Engine: JIT, defect, P5)

Description

[Octoploid]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0
Build ID: 20161217112731

Steps to reproduce:

Fixefox invokes undefined behavior in UB in js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14


Actual results:

Executing /home/markus/tmp/moz-build-dir/dist/bin/xpcshell -g /home/markus/tmp/moz-build-dir/dist/bin/ -a /home/markus/tmp/moz-build-dir/dist/bin/ -f /home/markus/tmp/gecko-dev/to
olkit/mozapps/installer/precompile_cache.js -e precompile_startupcache("resource://gre/");
/home/markus/tmp/gecko-dev/xpcom/components/nsComponentManager.cpp:313:9: runtime error: load of address 0x7f83710706a8 with insufficient space for an object of type 'const struct
 Module *'                                                                   
0x7f83710706a8: note: pointer points here
 00 00 00 00  c0 ba 73 5a 83 7f 00 00  40 02 74 5a 83 7f 00 00  60 2e 74 5a 83 7f 00 00  40 86 75 5a                                                                              
              ^ 
/home/markus/tmp/gecko-dev/js/src/jit/MoveResolver.h:179:7: runtime error: load of value 469, which is not a valid value for type 'Type'
/home/markus/tmp/gecko-dev/js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14: runtime error: store to misaligned address 0x7f83714ed25a for type 'uintptr_t', which requires 8 b
yte alignment
0x7f83714ed25a: note: pointer points here
 90 01  49 bb ff ff ff ff ff ff  ff ff 41 53 6a 00 85 c0  0f 84 2c 00 00 00 83 f8  01 0f 84 eb 00 00
              ^ 
/home/markus/tmp/gecko-dev/js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14: runtime error: store to misaligned address 0x7f83714f7e53 for type 'uintptr_t', which requires 8 b
yte alignment
0x7f83714f7e53: note: pointer points here
 4d  30 48 bf ff ff ff ff ff  ff ff ff 48 8b 3f ff 17  48 bf ff ff ff ff ff ff  ff ff 48 8b 3f ff 17
              ^ 
/home/markus/tmp/gecko-dev/js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14: runtime error: store to misaligned address 0x7f832e727b89 for type 'uintptr_t', which requires 8 b
yte alignment
0x7f832e727b89: note: pointer points here
 90 90 49  bb ff ff ff ff ff ff ff  ff 41 53 e8 48 59 dc 42  cc 6a 00 e9 65 00 00 00  68 20 30 00 00
              ^ 
/home/markus/tmp/gecko-dev/js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14: runtime error: store to misaligned address 0x7f832e727a77 for type 'uintptr_t', which requires 8 b
yte alignment
0x7f832e727a77: note: pointer points here
 00 00 48 ba ff  ff ff ff ff ff ff ff 83  7a 30 00 0f 84 2d 01 00  00 49 bb 00 00 00 00 00  80 fa ff
             ^ 
/home/markus/tmp/gecko-dev/js/src/jit/x86-shared/Assembler-x86-shared.h:3604:14: runtime error: store to misaligned address 0x7f832e72e673 for type 'uintptr_t', which requires 8 b
yte alignment
0x7f832e72e673: note: pointer points here
 8b  dc 49 bb de c0 ad de 00  00 00 00 41 53 49 bb 00  00 00 00 00 80 f8 ff 4c  0b d9 41 53 48 8b ec
              ^ 
resource://gre/components/BrowserElementParent.js
resource://gre/components/CSSUnprefixingService.js
resource://gre/components/ColorAnalyzer.js
resource://gre/components/ConsoleAPIStorage.js
resource://gre/components/ContentProcessSingleton.js
resource://gre/components/DOMSecureElement.js
resource://gre/components/DownloadLegacy.js
resource://gre/components/EditorUtils.js
resource://gre/components/FeedProcessor.js
resource://gre/components/FormAutofillContentService.js
resource://gre/components/FormAutofillStartup.js
resource://gre/components/FormHistoryStartup.js
resource://gre/components/FxAccountsPush.js
/home/markus/tmp/gecko-dev/js/src/vm/NativeObject.h:1088:72: runtime error: null pointer passed as argument 2, which is declared to never be null



Expected results:

xpcshell runs out of memory due to the UB above when compiled with gcc.

Comment 1

[Octoploid]

xpcshell keeps allocating memory:

Overhead  Shared Object  Symbol
  28.61%  libxul.so      [.] js::jit::MacroAssembler::PushRegsInMask
  15.39%  libxul.so      [.] js::jit::X86Encoding::BaseAssembler::X86InstructionFormatter::emitRex
  12.80%  libxul.so      [.] js::jit::AssemblerBuffer::ensureSpace
   9.73%  libxul.so      [.] js::PageProtectingVector<unsigned char, 256ul, js::SystemAllocPolicy, false, true, 32768ul>::unprotectUnused
   8.75%  libxul.so      [.] js::jit::GenericAssembler::spew

Comment 2

[Hannes Verschore [:h4writer]]

Which gcc version and configure flags are you using?

Comment 3

[Octoploid]

I'm using gcc trunk.

ac_add_options --enable-optimize=-O3
export CXXFLAGS="-flifetime-dse=1 -fno-delete-null-pointer-checks -march=native -fno-strict-aliasing -ffunction-sections -fdata-sections"

Comment 4

[Gian-Carlo Pascutto [:gcp]]

Compiler info was already provided a while ago by reporter.