1300716 - WebRtc - Memory corruption in webrtc::ThreadWindows::Run()

Status: RESOLVED DUPLICATE of bug 1293347

(Core :: WebRTC, defect, P1)

Description

[Looben Yang]

Attachment: webrtcThreadWindowsRun_Repro.html

Steps to reproduce:
1. Open webrtcThreadWindowsRun_Repro.html from a web server in Firefox.
2. Firefox crashes by executing corrupted address in webrtc::ThreadWindows::Run():


OS: Windows 10
Firefox version: 51.0a1 (2016-09-05)


(ab28.4696c): Access violation - code c0000005 (!!! second chance !!!)
eax=00000000 ebx=0e2ea6a0 ecx=6c9069fa edx=00000000 esi=0e2ea6a0 edi=5ada7bd4
eip=1268bb50 esp=f25ff9a8 ebp=f25ff9b8 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
1268bb50 20bb6812609a    and     byte ptr [ebx-659FED98h],bh ds:002b:a88eb908=00
0:2584> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************


FAULTING_IP: 
+cd64fa2a00000000
1268bb50 20bb6812609a    and     byte ptr [ebx-659FED98h],bh

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 1268bb50
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000008
   Parameter[1]: 1268bb50
Attempt to execute non-executable address 1268bb50

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=00000000 ebx=0e2ea6a0 ecx=6c9069fa edx=00000000 esi=0e2ea6a0 edi=5ada7bd4
eip=1268bb50 esp=f25ff9a8 ebp=f25ff9b8 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
1268bb50 20bb6812609a    and     byte ptr [ebx-659FED98h],bh ds:002b:a88eb908=00

FAULTING_THREAD:  0004696c

DEFAULT_BUCKET_ID:  SOFTWARE_NX_FAULT

PROCESS_NAME:  firefox.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000008

EXCEPTION_PARAMETER2:  1268bb50

WRITE_ADDRESS:  1268bb50 

FOLLOWUP_IP: 
xul!webrtc::ThreadWindows::Run+1f [c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\system_wrappers\source\thread_win.cc @ 136]
5ada7944 59              pop     ecx

FAILED_INSTRUCTION_ADDRESS: 
+6
1268bb50 20bb6812609a    and     byte ptr [ebx-659FED98h],bh

NTGLOBALFLAG:  0

APPLICATION_VERIFIER_FLAGS:  0

APP:  firefox.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

IP_ON_HEAP:  1268bb50
The fault address in not in any loaded module, please check your build's rebase
log at <releasedir>\bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.

PRIMARY_PROBLEM_CLASS:  SOFTWARE_NX_FAULT

BUGCHECK_STR:  APPLICATION_FAULT_SOFTWARE_NX_FAULT_INVALID_POINTER_READ

LAST_CONTROL_TRANSFER:  from 5ada7944 to 1268bb50

STACK_TEXT:  
WARNING: Frame IP not in any known module. Following frames may be wrong.
f25ff9a4 5ada7944 0da79a60 5ada7bd4 5ada7bdf 0x1268bb50
f25ff9b0 5ada7bdf f25ff9cc 760462c4 0e2ea6a0 xul!webrtc::ThreadWindows::Run+0x1f
f25ff9b8 760462c4 0e2ea6a0 760462a0 6f61274c xul!webrtc::ThreadWindows::StartThread+0xb
f25ff9cc 77270609 0e2ea6a0 6e5d095b 00000000 KERNEL32!BaseThreadInitThunk+0x24
f25ffa14 772705d4 ffffffff 7729254e 00000000 ntdll!__RtlUserThreadStart+0x2f
f25ffa24 00000000 5ada7bd4 0e2ea6a0 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

FAULTING_SOURCE_LINE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\system_wrappers\source\thread_win.cc

FAULTING_SOURCE_FILE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\system_wrappers\source\thread_win.cc

FAULTING_SOURCE_LINE_NUMBER:  136

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  xul!webrtc::ThreadWindows::Run+1f

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  57cd5710

FAILURE_BUCKET_ID:  SOFTWARE_NX_FAULT_c0000005_xul.dll!webrtc::ThreadWindows::Run

BUCKET_ID:  APPLICATION_FAULT_SOFTWARE_NX_FAULT_INVALID_POINTER_READ_BAD_IP_xul!webrtc::ThreadWindows::Run+1f

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:software_nx_fault_c0000005_xul.dll!webrtc::threadwindows::run

FAILURE_ID_HASH:  {7970fa2c-7623-db7f-b255-5a0278b5842d}

Followup: MachineOwner
---------

Comment 1

[Daniel Veditz [:dveditz]]

Wasn't able to reproduce on a Mac opt build (tried 50 and 51), but did get the following on the console:
"An attempt was made to use an object that is not, or is no longer, usable"

Might be windows-only, or maybe an ASAN build will surface where the real problem starts.

Comment 2

[Nils Ohlmeier [:drno]]

Randell ping me if I should try to repro on Win 10 and/or Linux ASAN tomorrow.

Comment 3

[Randell Jesup [:jesup] (needinfo me)]

I'm not seeing any crashes on linux-asan debug (inbound from a few days ago) or on win10 Nightly 51.  I am loading this locally.  I let it run for ~ 30 minutes, also randomly doing about:memory GC/CC/Minimize operations.

Comment 4

[Randell Jesup [:jesup] (needinfo me)]

Looben - can you reproduce this, or was it a one-time failure?  What OS? (Win10?)  Anything else that could help verify this?  Does it happen if you open the repro testcase from a local file?

Thanks

Comment 5

[Nils Ohlmeier [:drno]]

I'm trying right now on Win 10 by loading it from a real server on the Internet, but no luck to repro the problem so far.

Looben: is the web server running locally or on a real host on the Internet? I'm guessing this is pretty timing sensitive, so where the web server is located might matter for the timing here.

Another question: is "multi-process" turn on or off (on the General tab in about:preferences)? I notice that the console message is different depending on that setting (which makes sense as multi-process probably can change network timing quite a bit).

Comment 6

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #4)
> Looben - can you reproduce this, or was it a one-time failure?  What OS?
> (Win10?)  Anything else that could help verify this?  Does it happen if you
> open the repro testcase from a local file?
> 
> Thanks

yes, reproducible. not an one time failure. OS is Windows 10 as specified in the description.
Does not happen when opening it from a local file.

Comment 7

[Looben Yang]

(In reply to Nils Ohlmeier [:drno] from comment #5)
> I'm trying right now on Win 10 by loading it from a real server on the
> Internet, but no luck to repro the problem so far.
> 
> Looben: is the web server running locally or on a real host on the Internet?
> I'm guessing this is pretty timing sensitive, so where the web server is
> located might matter for the timing here.
> 
> Another question: is "multi-process" turn on or off (on the General tab in
> about:preferences)? I notice that the console message is different depending
> on that setting (which makes sense as multi-process probably can change
> network timing quite a bit).

Web server is run locally. I could have combine them into a single node.js script as before.
"multi-process" is off.

Comment 8

[Nils Ohlmeier [:drno]]

Thanks for the feedback Looben. I started a nodejs server locally on my Win 10 laptop and loaded the page from there, but within the first 30min no crash. I'll leave it up running over night and check back tomorrow morning.

Comment 9

[Nils Ohlmeier [:drno]]

No crash after running it for 18+ hours.
But I see a huge memory leak. I'll try chase to chase that leak and maybe we get an idea what could cause the crash.

Comment 10

[Looben Yang]

Sometimes the exact same test case crashes in AudioDeviceWindowsWave::DoGetCaptureVolumeThread():

Firefox version: 51.0a1 (2016-09-05)

(3588.307f4): Access violation - code c0000005 (!!! second chance !!!)
eax=00000102 ebx=f5101000 ecx=a27fe6d9 edx=00000000 esi=f5101000 edi=00008138
eip=5e3a4c15 esp=fff2fb00 ebp=fff2fb0c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x22:
5e3a4c15 8b06            mov     eax,dword ptr [esi]  ds:002b:f5101000=????????
0:2527> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************



Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.

*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\WINDOWS\SysWoW64\atidxx32.dll - 
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\WINDOWS\SysWoW64\SogouPy.ime - 

FAULTING_IP: 
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22 [c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc @ 400]
5e3a4c15 8b06            mov     eax,dword ptr [esi]

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 5e3a4c15 (xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x00000022)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: f5101000
Attempt to read from address f5101000

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=00000102 ebx=f5101000 ecx=a27fe6d9 edx=00000000 esi=f5101000 edi=00008138
eip=5e3a4c15 esp=fff2fb00 ebp=fff2fb0c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x22:
5e3a4c15 8b06            mov     eax,dword ptr [esi]  ds:002b:f5101000=????????

FAULTING_THREAD:  000307f4

PROCESS_NAME:  firefox.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  f5101000

READ_ADDRESS:  f5101000 

FOLLOWUP_IP: 
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22 [c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc @ 400]
5e3a4c15 8b06            mov     eax,dword ptr [esi]

NTGLOBALFLAG:  0

APPLICATION_VERIFIER_FLAGS:  0

APP:  firefox.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_READ_BEFORE_CALL

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ_BEFORE_CALL

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ_BEFORE_CALL

LAST_CONTROL_TRANSFER:  from 5e3a5f1e to 5e3a4c15

STACK_TEXT:  
fff2fb0c 5e3a5f1e fff2fb28 747562c4 f5101000 xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x22
fff2fb14 747562c4 f5101000 747562a0 a2f0b6a5 xul!webrtc::AudioDeviceWindowsWave::GetCaptureVolumeThread+0xb
fff2fb28 775b0609 f5101000 a1ea6968 00000000 KERNEL32!BaseThreadInitThunk+0x24
fff2fb70 775b05d4 ffffffff 775d2535 00000000 ntdll!__RtlUserThreadStart+0x2f
fff2fb80 00000000 5e3a5f13 f5101000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

FAULTING_SOURCE_LINE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc

FAULTING_SOURCE_FILE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc

FAULTING_SOURCE_LINE_NUMBER:  400

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  57cd5710

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_BEFORE_CALL_c0000005_xul.dll!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_READ_BEFORE_CALL_xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_before_call_c0000005_xul.dll!webrtc::audiodevicewindowswave::dogetcapturevolumethread

FAILURE_ID_HASH:  {b8fdf74b-5f15-4b7c-9a1e-f46ae10169c6}

Followup: MachineOwner
---------

Comment 11

[Nils Ohlmeier [:drno]]

Looking at the memory leaks I only saw the well known leaks around the nICEr registry which happen when terminating a PeerConnection. As the test creates and terminates PeerConnections at a very fast pace the leaked memory from that accumulates faster then normal. But I did not see any other leaks besides this known one.

Now the second crash signature from comment #10 looks like a thread shutdown problem when capturing sound on Windows around this http://searchfox.org/mozilla-central/source/media/webrtc/trunk/webrtc/modules/audio_device/win/audio_device_wave_win.cc#382
My guess would be that the switch statement in that function does not exit like it's suppose to do, and then the if condition in line 400 tries to read the _AGC variable which is no longer available. Randell what do you think?

Comment 12

[Looben Yang]

Attachment: repro2.html

The test case repro2.html before minimized seems a lot easier to reproduce. Would you have a try? It can be reproduced by openning the repro2.html locally.

Just ran it and I got:

Firefox nightly version: 51.0a1 (2016-09-11) (32-bit)

(5ea0.3f24): Access violation - code c0000005 (!!! second chance !!!)
eax=00000102 ebx=e7401000 ecx=0b6f2218 edx=00000000 esi=e7401000 edi=00008090
eip=10b5de46 esp=0010fc8c ebp=0010fc98 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x22:
10b5de46 8b06            mov     eax,dword ptr [esi]  ds:002b:e7401000=????????
0:2563> !analyze -v


FAULTING_IP: 
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22 [c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc @ 400]
10b5de46 8b06            mov     eax,dword ptr [esi]

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 10b5de46 (xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x00000022)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: e7401000
Attempt to read from address e7401000

FAULTING_THREAD:  00003f24

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ

PROCESS_NAME:  firefox.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  e7401000

READ_ADDRESS:  e7401000 

FOLLOWUP_IP: 
xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22 [c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc @ 400]
10b5de46 8b06            mov     eax,dword ptr [esi]

BUGCHECK_STR:  INVALID_POINTER_READ

NTGLOBALFLAG:  400

APPLICATION_VERIFIER_FLAGS:  0

APP:  firefox.exe

ANALYSIS_VERSION: 10.0.10240.9 x86fre

LAST_CONTROL_TRANSFER:  from 10b5f14f to 10b5de46

STACK_TEXT:  
0010fc98 10b5f14f 0010fcb4 75ba38f4 e7401000 xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+0x22
0010fca0 75ba38f4 e7401000 75ba38d0 08c55b20 xul!webrtc::AudioDeviceWindowsWave::GetCaptureVolumeThread+0xb
0010fcb4 77135de3 e7401000 0a1cd5df 00000000 KERNEL32!BaseThreadInitThunk+0x24
0010fcfc 77135dae ffffffff 7715b7bd 00000000 ntdll!__RtlUserThreadStart+0x2f
0010fd0c 00000000 10b5f144 e7401000 00000000 ntdll!_RtlUserThreadStart+0x1b


FAULTING_SOURCE_LINE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc

FAULTING_SOURCE_FILE:  c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\src\media\webrtc\trunk\webrtc\modules\audio_device\win\audio_device_wave_win.cc

FAULTING_SOURCE_LINE_NUMBER:  400

FAULTING_SOURCE_CODE:  
   396:                     "  unknown wait termination on get volume thread");
   397:                 return 1;
   398:         }
   399: 
>  400:         if (AGC())
   401:         {
   402:             uint32_t currentMicLevel = 0;
   403:             if (MicrophoneVolume(currentMicLevel) == 0)
   404:             {
   405:                 // This doesn't set the system volume, just stores it.


SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  57d556b1

STACK_COMMAND:  ~2563s ; kb

BUCKET_ID:  INVALID_POINTER_READ_xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ_xul!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread+22

FAILURE_PROBLEM_CLASS:  INVALID_POINTER_READ

FAILURE_EXCEPTION_CODE:  c0000005

FAILURE_IMAGE_NAME:  xul.dll

FAILURE_FUNCTION_NAME:  webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread

FAILURE_SYMBOL_NAME:  xul.dll!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_c0000005_xul.dll!webrtc::AudioDeviceWindowsWave::DoGetCaptureVolumeThread

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_c0000005_xul.dll!webrtc::audiodevicewindowswave::dogetcapturevolumethread

FAILURE_ID_HASH:  {e7729c00-942e-9b0c-933d-10c001998520}

Followup:     MachineOwner
---------

Comment 13

[Nils Ohlmeier [:drno]]

I ran the new repro2.html test case for a few hours on my Win 10 laptop without any crash.
My guess is that the whole crash is somehow related to the audio hardware or drivers on Looben's machine.

Comment 14

[Looben Yang]

(In reply to Nils Ohlmeier [:drno] from comment #13)
> I ran the new repro2.html test case for a few hours on my Win 10 laptop
> without any crash.
> My guess is that the whole crash is somehow related to the audio hardware or
> drivers on Looben's machine.

It's unlikely to be hardware or driver related, as I reproduced it in two different machines.

Comment 15

[Nils Ohlmeier [:drno]]

Looben as we aren't able to repro this right now, could you please check if you can repro this with Firefox 48 and 49?

Comment 16

[Nils Ohlmeier [:drno]]

Looben one more request: could you please check if the build from this try run  https://treeherder.mozilla.org/#/jobs?repo=try&revision=574fc9ca06bd fixes the problem for you?

It is a little bit vague guess right now, but might match.

Comment 17

[Randell Jesup [:jesup] (needinfo me)]

Actually, try a build from https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=fbdc035f911fa847084af3772dfcceddaf68927a which has the final fix for that bug.

Comment 18

[Looben Yang]

(In reply to Nils Ohlmeier [:drno] from comment #16)
> Looben one more request: could you please check if the build from this try
> run  https://treeherder.mozilla.org/#/jobs?repo=try&revision=574fc9ca06bd
> fixes the problem for you?
> 
> It is a little bit vague guess right now, but might match.

Reproduced it with repro2.html served from web server in this try build:

51.0a1 (2016-09-16) (32-bit)

(3b170.430a4): Access violation - code c0000005 (!!! second chance !!!)
eax=00000102 ebx=edc01000 ecx=67f183d1 edx=00000000 esi=edc01000 edi=00005988
eip=59f52b85 esp=0020fec4 ebp=0020fed0 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x433d3:
59f52b85 8b06            mov     eax,dword ptr [esi]  ds:002b:edc01000=????????
0:1621> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

*** ERROR: Symbol file could not be found.  Defaulted to export symbols for firefox.exe - 


Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.

*** ERROR: Symbol file could not be found.  Defaulted to export symbols for E:\ChromeBuilds\try\firefox\nss3.dll - 
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\WINDOWS\SysWoW64\atidxx32.dll - 
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\WINDOWS\SysWoW64\SogouPy.ime - 

FAULTING_IP: 
xul!mozilla::net::LoadInfo::SetVerifySignedContent+433d3
59f52b85 8b06            mov     eax,dword ptr [esi]

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 59f52b85 (xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x000433d3)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: edc01000
Attempt to read from address edc01000

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=00000102 ebx=edc01000 ecx=67f183d1 edx=00000000 esi=edc01000 edi=00005988
eip=59f52b85 esp=0020fec4 ebp=0020fed0 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x433d3:
59f52b85 8b06            mov     eax,dword ptr [esi]  ds:002b:edc01000=????????

FAULTING_THREAD:  000430a4

PROCESS_NAME:  firefox.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  edc01000

READ_ADDRESS:  edc01000 

FOLLOWUP_IP: 
xul!mozilla::net::LoadInfo::SetVerifySignedContent+433d3
59f52b85 8b06            mov     eax,dword ptr [esi]

NTGLOBALFLAG:  2000000

APPLICATION_VERIFIER_FLAGS:  0

APP:  firefox.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_READ_BEFORE_CALL

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ_BEFORE_CALL

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ_BEFORE_CALL

LAST_CONTROL_TRANSFER:  from 59f53ed0 to 59f52b85

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
0020fed0 59f53ed0 0020feec 74d162c4 edc01000 xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x433d3
0020fed8 74d162c4 edc01000 74d162a0 675dc008 xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x4471e
0020feec 77320609 edc01000 64b75f43 00000000 KERNEL32!BaseThreadInitThunk+0x24
0020ff34 773205d4 ffffffff 7734251b 00000000 ntdll!__RtlUserThreadStart+0x2f
0020ff44 00000000 59f53ec5 edc01000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  xul!mozilla::net::LoadInfo::SetVerifySignedContent+433d3

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  57dc5367

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_BEFORE_CALL_c0000005_xul.dll!mozilla::net::LoadInfo::SetVerifySignedContent

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_READ_BEFORE_CALL_xul!mozilla::net::LoadInfo::SetVerifySignedContent+433d3

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_before_call_c0000005_xul.dll!mozilla::net::loadinfo::setverifysignedcontent

FAILURE_ID_HASH:  {5f21be11-6dab-1dd8-7b43-918903f45b57}

Followup: MachineOwner
---------

Comment 19

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #17)
> Actually, try a build from
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> inbound&revision=fbdc035f911fa847084af3772dfcceddaf68927a which has the
> final fix for that bug.

Reproduced with this second try build too:

51.0a1 (2016-09-17) (32-bit)

(436b4.4ace8): Access violation - code c0000005 (!!! second chance !!!)
eax=00000102 ebx=ced01000 ecx=26392374 edx=00000000 esi=ced01000 edi=00006020
eip=59f12553 esp=00c0fb30 ebp=00c0fb3c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x1112b:
59f12553 8b06            mov     eax,dword ptr [esi]  ds:002b:ced01000=????????
0:1842> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

*** ERROR: Symbol file could not be found.  Defaulted to export symbols for firefox.exe - 
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for E:\ChromeBuilds\try\firefox\nss3.dll - 
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\WINDOWS\SysWoW64\atidxx32.dll - 

FAULTING_IP: 
xul!mozilla::net::LoadInfo::SetVerifySignedContent+1112b
59f12553 8b06            mov     eax,dword ptr [esi]

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 59f12553 (xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x0001112b)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: ced01000
Attempt to read from address ced01000

CONTEXT:  00000000 -- (.cxr 0x0;r)
eax=00000102 ebx=ced01000 ecx=26392374 edx=00000000 esi=ced01000 edi=00006020
eip=59f12553 esp=00c0fb30 ebp=00c0fb3c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x1112b:
59f12553 8b06            mov     eax,dword ptr [esi]  ds:002b:ced01000=????????

FAULTING_THREAD:  0004ace8

PROCESS_NAME:  firefox.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  ced01000

READ_ADDRESS:  ced01000 

FOLLOWUP_IP: 
xul!mozilla::net::LoadInfo::SetVerifySignedContent+1112b
59f12553 8b06            mov     eax,dword ptr [esi]

NTGLOBALFLAG:  2000000

APPLICATION_VERIFIER_FLAGS:  0

APP:  firefox.exe

ANALYSIS_VERSION: 6.3.9600.17029 (debuggers(dbg).140219-1702) x86fre

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_READ_BEFORE_CALL

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ_BEFORE_CALL

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ_BEFORE_CALL

LAST_CONTROL_TRANSFER:  from 59f1389e to 59f12553

STACK_TEXT:  
WARNING: Stack unwind information not available. Following frames may be wrong.
00c0fb3c 59f1389e 00c0fb58 74d162c4 ced01000 xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x1112b
00c0fb44 74d162c4 ced01000 74d162a0 2694bee0 xul!mozilla::net::LoadInfo::SetVerifySignedContent+0x12476
00c0fb58 77320609 ced01000 258e9c95 00000000 KERNEL32!BaseThreadInitThunk+0x24
00c0fba0 773205d4 ffffffff 7734251c 00000000 ntdll!__RtlUserThreadStart+0x2f
00c0fbb0 00000000 59f13893 ced01000 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  .cxr 0x0 ; kb

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  xul!mozilla::net::LoadInfo::SetVerifySignedContent+1112b

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  57dd0c3e

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_BEFORE_CALL_c0000005_xul.dll!mozilla::net::LoadInfo::SetVerifySignedContent

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_READ_BEFORE_CALL_xul!mozilla::net::LoadInfo::SetVerifySignedContent+1112b

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_before_call_c0000005_xul.dll!mozilla::net::loadinfo::setverifysignedcontent

FAILURE_ID_HASH:  {5f21be11-6dab-1dd8-7b43-918903f45b57}

Followup: MachineOwner
---------

Comment 20

[Looben Yang]

BTW. why try build no longer generates ASAN build? the asan build link has no binary:
Build: - linux64 - ( https://queue.taskcluster.net/v1/task/MBRKeSI-TcyMNxL1UA7idw/runs/0/artifacts/ )

Comment 21

[Nils Ohlmeier [:drno]]

(In reply to Looben Yang from comment #20)
> BTW. why try build no longer generates ASAN build? the asan build link has
> no binary:
> Build: - linux64 - (
> https://queue.taskcluster.net/v1/task/MBRKeSI-TcyMNxL1UA7idw/runs/0/
> artifacts/ )

Interesting. Hadn't noticed that. But I think it is probably more a treeherder front end issue, as the tests still got executed on the ASAN build. And the Linux debug build has the same problem. I guess the logic for locating build from within the tc() build step is broken. I'll have to check.

Thanks for testing it so quickly! Appreciated.

Comment 22

[Looben Yang]

Using the same test case webrtcThreadWindowsRun_Repro.html, just set the refresh timer longer as my Linux vertual machine is low (   125+Math.floor(200*Math.random())  -> 125+Math.floor(500*Math.random()) ), Got the following Use After Free report in Linux ASAN build.

Looks like a double free, and it's very common for double free issue to crash at different places if not exploited explicitly.
If you think it's a different bug, feel free to treat it in separate ticket.


51.0a1 (2016-08-10)


=================================================================
==16991==ERROR: AddressSanitizer: heap-use-after-free on address 0x611003767670 at pc 0x7f597d78984f bp 0x7f596ed47f80 sp 0x7f596ed47f78
READ of size 8 at 0x611003767670 thread T7 (Socket Thread)
    #0 0x7f597d78984e in sctp_iterator_inp_being_freed /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_pcb.c:3816:17
    #1 0x7f597d78984e in sctp_inpcb_free /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_pcb.c:3904
    #2 0x7f597d7af31e in sctp_close /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_usrreq.c:944:4
    #3 0x7f597d802816 in sofree /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/user_socket.c:256:2
    #4 0x7f597d80c30e in mozilla::DataChannelConnection::DestroyOnSTS(socket*, socket*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/datachannel/DataChannel.cpp:297:5
    #5 0x7f597d825950 in apply<RefPtr<mozilla::DataChannelConnection>, void (mozilla::DataChannelConnection::*)(socket *, socket *), socket *, socket *, 0, 1> /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dist/include/mtransport/runnable_utils.h:102:5
    #6 0x7f597d825950 in mozilla::runnable_args_memfn<RefPtr<mozilla::DataChannelConnection>, void (mozilla::DataChannelConnection::*)(socket*, socket*), socket*, socket*>::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dist/include/mtransport/runnable_utils.h:169
    #7 0x7f597cb47426 in nsThread::ProcessNextEvent(bool, bool*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThread.cpp:1058:7
    #8 0x7f597cbc53bc in NS_ProcessNextEvent(nsIThread*, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsThreadUtils.cpp:290:10
    #9 0x7f597cd91bee in mozilla::net::nsSocketTransportService::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsSocketTransportService2.cpp:901:21
    #10 0x7f597cd9441c in non-virtual thunk to mozilla::net::nsSocketTransportService::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsSocketTransportService2.cpp:787:27
    #11 0x7f597cb47426 in nsThread::ProcessNextEvent(bool, bool*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThread.cpp:1058:7
    #12 0x7f597cbc53bc in NS_ProcessNextEvent(nsIThread*, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsThreadUtils.cpp:290:10
    #13 0x7f597d8e519a in mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/glue/MessagePump.cpp:338:20
    #14 0x7f597d858b08 in RunInternal /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:232:3
    #15 0x7f597d858b08 in RunHandler /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:225
    #16 0x7f597d858b08 in MessageLoop::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:205
    #17 0x7f597cb42741 in nsThread::ThreadFunc(void*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThread.cpp:459:5
    #18 0x7f5993e07378 in _pt_root /builds/slave/m-cen-l64-asan-000000000000000/build/src/nsprpub/pr/src/pthreads/ptthread.c:216:5
    #19 0x7f5997388181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312
    #20 0x7f599647a47c in clone /build/buildd/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111

0x611003767670 is located 112 bytes inside of 200-byte region [0x611003767600,0x6110037676c8)
freed by thread T47 here:
    #0 0x4b215b in __interceptor_free /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:38:3
    #1 0x7f597d7db5f6 in sctp_iterator_work /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctputil.c:1323:3
    #2 0x7f597d7db5f6 in sctp_iterator_worker /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctputil.c:1472
    #3 0x7f597d6cbc83 in sctp_iterator_thread /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_bsd_addr.c:159:3
    #4 0x7f5997388181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312

previously allocated by thread T50 here:
    #0 0x4b247b in malloc /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:52:3
    #1 0x7f597d79c49c in sctp_initiate_iterator /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_pcb.c:8024:2
    #2 0x7f597d7dcf71 in sctp_handle_addr_wq /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctputil.c:1521:9
    #3 0x7f597d7dcf71 in sctp_timeout_handler /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctputil.c:1693
    #4 0x7f597d6cccac in sctp_handle_tick /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_callout.c:155:4
    #5 0x7f597d6cccac in user_sctp_timer_iterate /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_callout.c:194
    #6 0x7f5997388181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312

Thread T7 (Socket Thread) created by T0 here:
    #0 0x49a839 in __interceptor_pthread_create /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:238:3
    #1 0x7f5993e03f3f in _PR_CreateThread /builds/slave/m-cen-l64-asan-000000000000000/build/src/nsprpub/pr/src/pthreads/ptthread.c:457:14
    #2 0x7f5993e03b4a in PR_CreateThread /builds/slave/m-cen-l64-asan-000000000000000/build/src/nsprpub/pr/src/pthreads/ptthread.c:548:12
    #3 0x7f597cb43ecb in nsThread::Init() /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThread.cpp:630:8
    #4 0x7f597cb4b5ef in nsThreadManager::NewThread(unsigned int, unsigned int, nsIThread**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThreadManager.cpp:253:17
    #5 0x7f597cbc43ae in NS_NewThread(nsIThread**, nsIRunnable*, unsigned int) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsThreadUtils.cpp:64:5
    #6 0x7f597cd8f0e8 in nsresult NS_NewNamedThread<14ul>(char const (&) [14ul], nsIThread**, nsIRunnable*, unsigned int) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dist/include/nsThreadUtils.h:79:17
    #7 0x7f597cd8e7ef in mozilla::net::nsSocketTransportService::Init() /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsSocketTransportService2.cpp:523:19
    #8 0x7f597d82d9cc in nsSocketTransportServiceConstructor(nsISupports*, nsID const&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/build/nsNetModule.cpp:80:1
    #9 0x7f597cb18471 in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:1160:10
    #10 0x7f597cb0f882 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:1516:10
    #11 0x7f597cbafa9a in CallGetService /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsComponentManagerUtils.cpp:67:10
    #12 0x7f597cbafa9a in nsGetServiceByContractIDWithError::operator()(nsID const&, void**) const /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsComponentManagerUtils.cpp:292
    #13 0x7f597cba5ee0 in nsCOMPtr_base::assign_from_gs_contractid_with_error(nsGetServiceByContractIDWithError const&, nsID const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsCOMPtr.cpp:106:7
    #14 0x7f597ccf428f in operator= /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dist/include/nsCOMPtr.h:644:5
    #15 0x7f597ccf428f in InitializeSocketTransportService /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsIOService.cpp:297
    #16 0x7f597ccf428f in mozilla::net::nsIOService::SetOffline(bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsIOService.cpp:1076
    #17 0x7f597ccf313d in mozilla::net::nsIOService::Init() /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsIOService.cpp:264:5
    #18 0x7f597ccf5a03 in mozilla::net::nsIOService::GetInstance() /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsIOService.cpp:349:23
    #19 0x7f597d82d737 in nsIOServiceConstructor(nsISupports*, nsID const&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/build/nsNetModule.cpp:62:1
    #20 0x7f597cb18471 in nsComponentManagerImpl::CreateInstanceByContractID(char const*, nsISupports*, nsID const&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:1160:10
    #21 0x7f597cb0f882 in nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:1516:10
    #22 0x7f597cbafa01 in CallGetService /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsComponentManagerUtils.cpp:67:10
    #23 0x7f597cbafa01 in nsGetServiceByContractID::operator()(nsID const&, void**) const /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsComponentManagerUtils.cpp:280
    #24 0x7f597cb95073 in assign_from_gs_contractid /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsCOMPtr.cpp:95:7
    #25 0x7f597cb95073 in nsCOMPtr /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsCOMPtr.h:539
    #26 0x7f597cb95073 in mozilla::services::GetIOService() /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/build/ServiceList.h:16
    #27 0x7f597cd27c35 in do_GetIOService(nsresult*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsNetUtilInlines.h:46:33
    #28 0x7f597cd2825f in net_EnsureIOService /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsNetUtilInlines.h:86:16
    #29 0x7f597cd2825f in NS_NewURI(nsIURI**, nsACString_internal const&, char const*, nsIURI*, nsIIOService*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/base/nsNetUtilInlines.h:113
    #30 0x7f597cb80706 in GetManifestURI /builds/slave/m-cen-l64-asan-000000000000000/build/src/chrome/nsChromeRegistryChrome.cpp:675:5
    #31 0x7f597cb80706 in nsChromeRegistry::ManifestProcessingContext::ResolveURI(char const*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/chrome/nsChromeRegistryChrome.cpp:692
    #32 0x7f597cb81c83 in nsChromeRegistryChrome::ManifestLocale(nsChromeRegistry::ManifestProcessingContext&, int, char* const*, int) /builds/slave/m-cen-l64-asan-000000000000000/build/src/chrome/nsChromeRegistryChrome.cpp:773:31
    #33 0x7f597cb22024 in ParseManifest(NSLocationType, mozilla::FileLocation&, char*, bool, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/ManifestParser.cpp:771:7
    #34 0x7f597cb13539 in DoRegisterManifest /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:595:5
    #35 0x7f597cb13539 in nsComponentManagerImpl::RegisterManifest(NSLocationType, mozilla::FileLocation&, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:608
    #36 0x7f597cb138b3 in nsComponentManagerImpl::ManifestManifest(nsComponentManagerImpl::ManifestProcessingContext&, int, char* const*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:617:3
    #37 0x7f597cb222ed in ParseManifest(NSLocationType, mozilla::FileLocation&, char*, bool, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/ManifestParser.cpp:780:9
    #38 0x7f597cb13539 in DoRegisterManifest /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:595:5
    #39 0x7f597cb13539 in nsComponentManagerImpl::RegisterManifest(NSLocationType, mozilla::FileLocation&, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:608
    #40 0x7f597cb1120c in RereadChromeManifests /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:794:5
    #41 0x7f597cb1120c in nsComponentManagerImpl::Init() /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/components/nsComponentManager.cpp:398
    #42 0x7f597cb99929 in NS_InitXPCOM2 /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/build/XPCOMInit.cpp:713:8
    #43 0x7f598545729b in Initialize /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:1399:8
    #44 0x7f598545729b in XREMain::XRE_main(int, char**, nsXREAppData const*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4402
    #45 0x7f598545820a in XRE_main /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4497:16
    #46 0x4df89a in do_main /builds/slave/m-cen-l64-asan-000000000000000/build/src/browser/app/nsBrowserApp.cpp:247:10
    #47 0x4df89a in main /builds/slave/m-cen-l64-asan-000000000000000/build/src/browser/app/nsBrowserApp.cpp:380
    #48 0x7f59963a1ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287

Thread T47 created by T0 here:
    #0 0x49a839 in __interceptor_pthread_create /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:238:3
    #1 0x7f597d6cbba9 in sctp_startup_iterator /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_bsd_addr.c:207:6
    #2 0x7f597d797537 in sctp_pcb_init /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_pcb.c:6818:2
    #3 0x7f597d7add28 in sctp_init /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_usrreq.c:170:2
    #4 0x7f597d80c4f1 in mozilla::DataChannelConnection::Init(unsigned short, unsigned short, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/datachannel/DataChannel.cpp:338:9
    #5 0x7f597e7cdeeb in mozilla::PeerConnectionImpl::EnsureDataConnection(unsigned short) /builds/slave/m-cen-l64-asan-000000000000000/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1155:8
    #6 0x7f597e7cf6ca in mozilla::PeerConnectionImpl::CreateDataChannel(nsAString_internal const&, nsAString_internal const&, unsigned short, bool, unsigned short, unsigned short, bool, unsigned short, nsDOMDataChannel**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1369:17
    #7 0x7f597e7cf4a9 in mozilla::PeerConnectionImpl::CreateDataChannel(nsAString_internal const&, nsAString_internal const&, unsigned short, bool, unsigned short, unsigned short, bool, unsigned short, mozilla::ErrorResult&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1341:8
    #8 0x7f598035e446 in mozilla::dom::PeerConnectionImplBinding::createDataChannel(JSContext*, JS::Handle<JSObject*>, mozilla::PeerConnectionImpl*, JSJitMethodCallArgs const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/PeerConnectionImplBinding.cpp:1310:48
    #9 0x7f59816dcea7 in mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/bindings/BindingUtils.cpp:2812:13
    #10 0x7f5987683ac3 in CallJSNative /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jscntxtinlines.h:235:15
    #11 0x7f5987683ac3 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:441
    #12 0x7f5987663eeb in CallFromStack /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:504:12
    #13 0x7f5987663eeb in Interpret(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:2873
    #14 0x7f59876499f5 in js::RunScript(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:399:12
    #15 0x7f5987684323 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:471:15
    #16 0x7f5987684db1 in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:517:10
    #17 0x7f59871b8c58 in JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:2852:12
    #18 0x7f598048f04b in mozilla::dom::RTCPeerConnectionJSImpl::CreateDataChannel(nsAString_internal const&, mozilla::dom::RTCDataChannelInit const&, mozilla::ErrorResult&, JSCompartment*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/RTCPeerConnectionBinding.cpp:7587:8
    #19 0x7f5980592be7 in CreateDataChannel /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/RTCPeerConnectionBinding.cpp:9128:10
    #20 0x7f5980592be7 in mozilla::dom::RTCPeerConnectionBinding::createDataChannel(JSContext*, JS::Handle<JSObject*>, mozilla::dom::RTCPeerConnection*, JSJitMethodCallArgs const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/RTCPeerConnectionBinding.cpp:4552
    #21 0x7f59816dcea7 in mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/bindings/BindingUtils.cpp:2812:13
    #22 0x7f5987683ac3 in CallJSNative /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jscntxtinlines.h:235:15
    #23 0x7f5987683ac3 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:441
    #24 0x7f5987663eeb in CallFromStack /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:504:12
    #25 0x7f5987663eeb in Interpret(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:2873
    #26 0x7f59876499f5 in js::RunScript(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:399:12
    #27 0x7f5987686737 in js::ExecuteKernel(JSContext*, JS::Handle<JSScript*>, JSObject&, JS::Value const&, js::AbstractFramePtr, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:679:15
    #28 0x7f5987686e6e in js::Execute(JSContext*, JS::Handle<JSScript*>, JSObject&, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:711:12
    #29 0x7f59871cab5d in Evaluate(JSContext*, JS::Handle<JSObject*>, JS::Handle<js::StaticScope*>, JS::ReadOnlyCompileOptions const&, JS::SourceBufferHolder&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:4437:19
    #30 0x7f59871cb6d1 in Evaluate /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:4464:12
    #31 0x7f59871cb6d1 in JS::Evaluate(JSContext*, JS::AutoVectorRooter<JSObject*>&, JS::ReadOnlyCompileOptions const&, JS::SourceBufferHolder&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:4525
    #32 0x7f597f9590ed in nsJSUtils::EvaluateString(JSContext*, JS::SourceBufferHolder&, JS::Handle<JSObject*>, JS::CompileOptions&, nsJSUtils::EvaluateOptions const&, JS::MutableHandle<JS::Value>, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsJSUtils.cpp:206:12
    #33 0x7f597f959bdf in nsJSUtils::EvaluateString(JSContext*, JS::SourceBufferHolder&, JS::Handle<JSObject*>, JS::CompileOptions&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsJSUtils.cpp:266:10
    #34 0x7f597f9e3cd7 in nsScriptLoader::EvaluateScript(nsScriptLoadRequest*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptLoader.cpp:2037:12
    #35 0x7f597f9e0b2a in nsScriptLoader::ProcessRequest(nsScriptLoadRequest*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptLoader.cpp:1836:10
    #36 0x7f597f9ca40e in nsScriptLoader::ProcessScriptElement(nsIScriptElement*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptLoader.cpp:1574:10
    #37 0x7f597f9c6ba2 in nsScriptElement::MaybeProcessScript() /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptElement.cpp:141:10
    #38 0x7f597eafca14 in AttemptToExecute /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsIScriptElement.h:222:18
    #39 0x7f597eafca14 in nsHtml5TreeOpExecutor::RunScript(nsIContent*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:664
    #40 0x7f597eafb1e1 in nsHtml5TreeOpExecutor::RunFlushLoop() /builds/slave/m-cen-l64-asan-000000000000000/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:488:7
    #41 0x7f597eaffa3b in nsHtml5ExecutorFlusher::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/parser/html/nsHtml5StreamParser.cpp:128:9
    #42 0x7f597cb47426 in nsThread::ProcessNextEvent(bool, bool*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThread.cpp:1058:7
    #43 0x7f597cbc53bc in NS_ProcessNextEvent(nsIThread*, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsThreadUtils.cpp:290:10
    #44 0x7f597d8e3bff in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/glue/MessagePump.cpp:96:21
    #45 0x7f597d858b08 in RunInternal /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:232:3
    #46 0x7f597d858b08 in RunHandler /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:225
    #47 0x7f597d858b08 in MessageLoop::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:205
    #48 0x7f59833a65af in nsBaseAppShell::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/widget/nsBaseAppShell.cpp:156:3
    #49 0x7f5985303a71 in nsAppStartup::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/components/startup/nsAppStartup.cpp:284:19
    #50 0x7f5985455dae in XREMain::XRE_mainRun() /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4279:10
    #51 0x7f598545733e in XREMain::XRE_main(int, char**, nsXREAppData const*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4406:8
    #52 0x7f598545820a in XRE_main /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4497:16
    #53 0x4df89a in do_main /builds/slave/m-cen-l64-asan-000000000000000/build/src/browser/app/nsBrowserApp.cpp:247:10
    #54 0x4df89a in main /builds/slave/m-cen-l64-asan-000000000000000/build/src/browser/app/nsBrowserApp.cpp:380
    #55 0x7f59963a1ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287

Thread T50 created by T0 here:
    #0 0x49a839 in __interceptor_pthread_create /builds/slave/moz-toolchain/src/llvm/projects/compiler-rt/lib/asan/asan_interceptors.cc:238:3
    #1 0x7f597d6cceaa in sctp_start_timer /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_callout.c:213:7
    #2 0x7f597d80c4f1 in mozilla::DataChannelConnection::Init(unsigned short, unsigned short, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/datachannel/DataChannel.cpp:338:9
    #3 0x7f597e7cdeeb in mozilla::PeerConnectionImpl::EnsureDataConnection(unsigned short) /builds/slave/m-cen-l64-asan-000000000000000/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1155:8
    #4 0x7f597e7cf6ca in mozilla::PeerConnectionImpl::CreateDataChannel(nsAString_internal const&, nsAString_internal const&, unsigned short, bool, unsigned short, unsigned short, bool, unsigned short, nsDOMDataChannel**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1369:17
    #5 0x7f597e7cf4a9 in mozilla::PeerConnectionImpl::CreateDataChannel(nsAString_internal const&, nsAString_internal const&, unsigned short, bool, unsigned short, unsigned short, bool, unsigned short, mozilla::ErrorResult&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp:1341:8
    #6 0x7f598035e446 in mozilla::dom::PeerConnectionImplBinding::createDataChannel(JSContext*, JS::Handle<JSObject*>, mozilla::PeerConnectionImpl*, JSJitMethodCallArgs const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/PeerConnectionImplBinding.cpp:1310:48
    #7 0x7f59816dcea7 in mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/bindings/BindingUtils.cpp:2812:13
    #8 0x7f5987683ac3 in CallJSNative /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jscntxtinlines.h:235:15
    #9 0x7f5987683ac3 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:441
    #10 0x7f5987663eeb in CallFromStack /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:504:12
    #11 0x7f5987663eeb in Interpret(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:2873
    #12 0x7f59876499f5 in js::RunScript(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:399:12
    #13 0x7f5987684323 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:471:15
    #14 0x7f5987684db1 in js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:517:10
    #15 0x7f59871b8c58 in JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:2852:12
    #16 0x7f598048f04b in mozilla::dom::RTCPeerConnectionJSImpl::CreateDataChannel(nsAString_internal const&, mozilla::dom::RTCDataChannelInit const&, mozilla::ErrorResult&, JSCompartment*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/RTCPeerConnectionBinding.cpp:7587:8
    #17 0x7f5980592be7 in CreateDataChannel /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/RTCPeerConnectionBinding.cpp:9128:10
    #18 0x7f5980592be7 in mozilla::dom::RTCPeerConnectionBinding::createDataChannel(JSContext*, JS::Handle<JSObject*>, mozilla::dom::RTCPeerConnection*, JSJitMethodCallArgs const&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/obj-firefox/dom/bindings/RTCPeerConnectionBinding.cpp:4552
    #19 0x7f59816dcea7 in mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/bindings/BindingUtils.cpp:2812:13
    #20 0x7f5987683ac3 in CallJSNative /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jscntxtinlines.h:235:15
    #21 0x7f5987683ac3 in js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:441
    #22 0x7f5987663eeb in CallFromStack /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:504:12
    #23 0x7f5987663eeb in Interpret(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:2873
    #24 0x7f59876499f5 in js::RunScript(JSContext*, js::RunState&) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:399:12
    #25 0x7f5987686737 in js::ExecuteKernel(JSContext*, JS::Handle<JSScript*>, JSObject&, JS::Value const&, js::AbstractFramePtr, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:679:15
    #26 0x7f5987686e6e in js::Execute(JSContext*, JS::Handle<JSScript*>, JSObject&, JS::Value*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/vm/Interpreter.cpp:711:12
    #27 0x7f59871cab5d in Evaluate(JSContext*, JS::Handle<JSObject*>, JS::Handle<js::StaticScope*>, JS::ReadOnlyCompileOptions const&, JS::SourceBufferHolder&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:4437:19
    #28 0x7f59871cb6d1 in Evaluate /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:4464:12
    #29 0x7f59871cb6d1 in JS::Evaluate(JSContext*, JS::AutoVectorRooter<JSObject*>&, JS::ReadOnlyCompileOptions const&, JS::SourceBufferHolder&, JS::MutableHandle<JS::Value>) /builds/slave/m-cen-l64-asan-000000000000000/build/src/js/src/jsapi.cpp:4525
    #30 0x7f597f9590ed in nsJSUtils::EvaluateString(JSContext*, JS::SourceBufferHolder&, JS::Handle<JSObject*>, JS::CompileOptions&, nsJSUtils::EvaluateOptions const&, JS::MutableHandle<JS::Value>, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsJSUtils.cpp:206:12
    #31 0x7f597f959bdf in nsJSUtils::EvaluateString(JSContext*, JS::SourceBufferHolder&, JS::Handle<JSObject*>, JS::CompileOptions&, void**) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsJSUtils.cpp:266:10
    #32 0x7f597f9e3cd7 in nsScriptLoader::EvaluateScript(nsScriptLoadRequest*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptLoader.cpp:2037:12
    #33 0x7f597f9e0b2a in nsScriptLoader::ProcessRequest(nsScriptLoadRequest*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptLoader.cpp:1836:10
    #34 0x7f597f9ca40e in nsScriptLoader::ProcessScriptElement(nsIScriptElement*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptLoader.cpp:1574:10
    #35 0x7f597f9c6ba2 in nsScriptElement::MaybeProcessScript() /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsScriptElement.cpp:141:10
    #36 0x7f597eafca14 in AttemptToExecute /builds/slave/m-cen-l64-asan-000000000000000/build/src/dom/base/nsIScriptElement.h:222:18
    #37 0x7f597eafca14 in nsHtml5TreeOpExecutor::RunScript(nsIContent*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:664
    #38 0x7f597eafb1e1 in nsHtml5TreeOpExecutor::RunFlushLoop() /builds/slave/m-cen-l64-asan-000000000000000/build/src/parser/html/nsHtml5TreeOpExecutor.cpp:488:7
    #39 0x7f597eaffa3b in nsHtml5ExecutorFlusher::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/parser/html/nsHtml5StreamParser.cpp:128:9
    #40 0x7f597cb47426 in nsThread::ProcessNextEvent(bool, bool*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/threads/nsThread.cpp:1058:7
    #41 0x7f597cbc53bc in NS_ProcessNextEvent(nsIThread*, bool) /builds/slave/m-cen-l64-asan-000000000000000/build/src/xpcom/glue/nsThreadUtils.cpp:290:10
    #42 0x7f597d8e3bff in mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/glue/MessagePump.cpp:96:21
    #43 0x7f597d858b08 in RunInternal /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:232:3
    #44 0x7f597d858b08 in RunHandler /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:225
    #45 0x7f597d858b08 in MessageLoop::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/ipc/chromium/src/base/message_loop.cc:205
    #46 0x7f59833a65af in nsBaseAppShell::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/widget/nsBaseAppShell.cpp:156:3
    #47 0x7f5985303a71 in nsAppStartup::Run() /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/components/startup/nsAppStartup.cpp:284:19
    #48 0x7f5985455dae in XREMain::XRE_mainRun() /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4279:10
    #49 0x7f598545733e in XREMain::XRE_main(int, char**, nsXREAppData const*) /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4406:8
    #50 0x7f598545820a in XRE_main /builds/slave/m-cen-l64-asan-000000000000000/build/src/toolkit/xre/nsAppRunner.cpp:4497:16
    #51 0x4df89a in do_main /builds/slave/m-cen-l64-asan-000000000000000/build/src/browser/app/nsBrowserApp.cpp:247:10
    #52 0x4df89a in main /builds/slave/m-cen-l64-asan-000000000000000/build/src/browser/app/nsBrowserApp.cpp:380
    #53 0x7f59963a1ec4 in __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:287

SUMMARY: AddressSanitizer: heap-use-after-free /builds/slave/m-cen-l64-asan-000000000000000/build/src/netwerk/sctp/src/netinet/sctp_pcb.c:3816:17 in sctp_iterator_inp_being_freed
Shadow bytes around the buggy address:
  0x0c22806e4e70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c22806e4e80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c22806e4e90: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c22806e4ea0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c22806e4eb0: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fa fa
=>0x0c22806e4ec0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd
  0x0c22806e4ed0: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
  0x0c22806e4ee0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c22806e4ef0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c22806e4f00: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c22806e4f10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==16991==ABORTING

Comment 23

[Randell Jesup [:jesup] (needinfo me)]

Michael: see comment 22 re a double-free in sctp

Comment 24

[Randell Jesup [:jesup] (needinfo me)]

Byron - you should check this; note the free location (sctputil.c:1323)

Comment 25

[Randell Jesup [:jesup] (needinfo me)]

Preliminary shot at the iterator issue; does this affect either symptom (original problem or the SCTP UAF)?  Thanks for testing those - was worth checking.    Here's to hoping this fixes both; there's a chance.

Comment 26

[Randell Jesup [:jesup] (needinfo me)]

Helps if I give the link:  https://treeherder.mozilla.org/#/jobs?repo=try&revision=6fdd5a2cd686

Comment 27

[Michael Tüxen]

Does https://bugzilla.mozilla.org/show_bug.cgi?id=1293347 fix it?

Comment 28

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #26)
> Helps if I give the link: 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=6fdd5a2cd686

Could you also build win32 version?

Comment 29

[Randell Jesup [:jesup] (needinfo me)]

Triggered; they'll take an hour or two to build.

Comment 30

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Michael Tüxen from comment #27)
> Does https://bugzilla.mozilla.org/show_bug.cgi?id=1293347 fix it?

No, since that was already in the previous Nightlies (and Tries) that the reporter reproduced with.

I am concerned there are two places where it *may* leave a stale ptr behind after freeing the iterator; see the patch from the Try https://hg.mozilla.org/try/rev/ce6274c267b142ef28c37ccf637c1642977b254d

Comment 31

[Michael Tüxen]

Ahh, I see. Let me know if it fixes the issue.

Comment 32

[Byron Campen [:bwc]]

(In reply to Randell Jesup [:jesup] from comment #24)
> Byron - you should check this; note the free location (sctputil.c:1323)

The crash in comment 22 looks like bug 1293347.

Comment 33

[Byron Campen [:bwc]]

(In reply to Randell Jesup [:jesup] from comment #30)
> (In reply to Michael Tüxen from comment #27)
> > Does https://bugzilla.mozilla.org/show_bug.cgi?id=1293347 fix it?
> 
> No, since that was already in the previous Nightlies (and Tries) that the
> reporter reproduced with.
> 
> I am concerned there are two places where it *may* leave a stale ptr behind
> after freeing the iterator; see the patch from the Try
> https://hg.mozilla.org/try/rev/ce6274c267b142ef28c37ccf637c1642977b254d

Did reporter get the same crash as comment 22 on a later build? Because 8/10 was before any fix for bug 1293347 was landed.

Comment 34

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Byron Campen [:bwc] from comment #33)
> (In reply to Randell Jesup [:jesup] from comment #30)
> > (In reply to Michael Tüxen from comment #27)
> > > Does https://bugzilla.mozilla.org/show_bug.cgi?id=1293347 fix it?
> > 
> > No, since that was already in the previous Nightlies (and Tries) that the
> > reporter reproduced with.
> > 
> > I am concerned there are two places where it *may* leave a stale ptr behind
> > after freeing the iterator; see the patch from the Try
> > https://hg.mozilla.org/try/rev/ce6274c267b142ef28c37ccf637c1642977b254d
> 
> Did reporter get the same crash as comment 22 on a later build? Because 8/10
> was before any fix for bug 1293347 was landed.

Aha. looking at comment 22, the build is from 8-10:
51.0a1 (2016-08-10)
I thought he was testing with my Try build, or a newer Nightly.  That does look like the crash from bug 1293347, which hit inbound on 8/17, and central on 8-19 (would have been in 8/20 Nightly).

Byron and Michael, please look at my changeset above and see if that's something we need to add, or is that guaranteed in some other manner?  (I'm tempted to land it anyways as a safety-valve to ensure no stale pointers in case there's a change in invariants.)

Looben - since the sctp crash appears to be a bug already fixed in the other builds you're testing, it is a separate (fixed) issue.

Comment 35

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #29)
> Triggered; they'll take an hour or two to build.

Reproduced with this try build ( https://treeherder.mozilla.org/#/jobs?repo=try&revision=6fdd5a2cd686&selectedJob=27651908 ):


51.0a1 (2016-09-19) (32-bit)

FAULTING_IP: 
xul!mozilla::net::LoadInfo::SetEnforceSRI+42c80
109d4e93 8b06            mov     eax,dword ptr [esi]

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 109d4e93 (xul!mozilla::net::LoadInfo::SetEnforceSRI+0x00042c80)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: df201000
Attempt to read from address df201000

FAULTING_THREAD:  00008a40

DEFAULT_BUCKET_ID:  INVALID_POINTER_READ

PROCESS_NAME:  firefox.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s.

EXCEPTION_PARAMETER1:  00000000

EXCEPTION_PARAMETER2:  df201000

READ_ADDRESS:  df201000 

FOLLOWUP_IP: 
xul!mozilla::net::LoadInfo::SetEnforceSRI+42c80
109d4e93 8b06            mov     eax,dword ptr [esi]

BUGCHECK_STR:  INVALID_POINTER_READ

NTGLOBALFLAG:  400

APPLICATION_VERIFIER_FLAGS:  0

APP:  firefox.exe

ANALYSIS_VERSION: 10.0.10240.9 x86fre

LAST_CONTROL_TRANSFER:  from 109d61de to 109d4e93

STACK_TEXT:  
WARNING: Stack pointer is outside the normal stack bounds. Stack unwinding can be inaccurate.
WARNING: Stack unwind information not available. Following frames may be wrong.
ffc5fe5c 109d61de ffc5fe78 741138f4 df201000 xul!mozilla::net::LoadInfo::SetEnforceSRI+0x42c80
ffc5fe64 741138f4 df201000 741138d0 7856ad47 xul!mozilla::net::LoadInfo::SetEnforceSRI+0x43fcb
ffc5fe78 77125de3 df201000 7b50f053 00000000 KERNEL32!BaseThreadInitThunk+0x24
ffc5fec0 77125dae ffffffff 7714b7e6 00000000 ntdll!__RtlUserThreadStart+0x2f
ffc5fed0 00000000 109d61d3 df201000 00000000 ntdll!_RtlUserThreadStart+0x1b


SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  xul!mozilla::net::LoadInfo::SetEnforceSRI+42c80

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: xul

IMAGE_NAME:  xul.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  57e00b43

STACK_COMMAND:  ~2565s ; kb

BUCKET_ID:  INVALID_POINTER_READ_xul!mozilla::net::LoadInfo::SetEnforceSRI+42c80

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_READ_xul!mozilla::net::LoadInfo::SetEnforceSRI+42c80

FAILURE_PROBLEM_CLASS:  INVALID_POINTER_READ

FAILURE_EXCEPTION_CODE:  c0000005

FAILURE_IMAGE_NAME:  xul.dll

FAILURE_FUNCTION_NAME:  mozilla::net::LoadInfo::SetEnforceSRI

FAILURE_SYMBOL_NAME:  xul.dll!mozilla::net::LoadInfo::SetEnforceSRI

FAILURE_BUCKET_ID:  INVALID_POINTER_READ_c0000005_xul.dll!mozilla::net::LoadInfo::SetEnforceSRI

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:invalid_pointer_read_c0000005_xul.dll!mozilla::net::loadinfo::setenforcesri

FAILURE_ID_HASH:  {a08af55d-f9b3-8b10-7d06-ca34441c0732}

Followup:     MachineOwner
---------

WARNING: Stack pointer is outside the normal stack bounds. Stack unwinding can be inaccurate.
WARNING: Stack pointer is outside the normal stack bounds. Stack unwinding can be inaccurate.
0:2565> g
(6390.8a40): Access violation - code c0000005 (!!! second chance !!!)
WARNING: Stack pointer is outside the normal stack bounds. Stack unwinding can be inaccurate.
eax=00000102 ebx=df201000 ecx=7878d278 edx=00000000 esi=df201000 edi=0000867c
eip=109d4e93 esp=ffc5fe50 ebp=ffc5fe5c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
xul!mozilla::net::LoadInfo::SetEnforceSRI+0x42c80:
109d4e93 8b06            mov     eax,dword ptr [esi]  ds:002b:df201000=????????

Comment 36

[Looben Yang]

Attachment: server2.js

Comment 37

[Looben Yang]

Attachment: server_webrtcThreadWindowsRun_Repro.js

To make sure that we run the test case in the same way, I just uploaded the node.js servers I used in my testing. Basically the same steps as usual.

1. Put server2.js and repro2.html in the same location. Run server side script server2.js in Node.js (node server2.js ).
2. Enter http://localhost:12345 in Firefox browser.

Comment 38

[Randell Jesup [:jesup] (needinfo me)]

> To make sure that we run the test case in the same way, I just uploaded the
> node.js servers I used in my testing. Basically the same steps as usual.

Thanks, that may help a lot in reproducing as this seems rather timing dependent.

Comment 39

[Michael Tüxen]

Was this tested with the additional suggested changes from Randell?

Comment 40

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Michael Tüxen from comment #39)
> Was this tested with the additional suggested changes from Randell?

Yes, in comment 35 he links to the build used to test, which includes that patch

Comment 41

[Michael Tüxen]

OK. Is it possible to figure out which field of which variable was overwritten? I can't get that
much information from the above stack traces...

Comment 42

[Byron Campen [:bwc]]

> Byron and Michael, please look at my changeset above and see if that's
> something we need to add, or is that guaranteed in some other manner?  (I'm
> tempted to land it anyways as a safety-valve to ensure no stale pointers in
> case there's a change in invariants.)

Well, the first hunk in there might be ok, although the second hunk is modifying |sctp_it_ctl.cur_it| without acquiring the SCTP_ITERATOR_LOCK.

Comment 43

[Randell Jesup [:jesup] (needinfo me)]

(In reply to Byron Campen [:bwc] from comment #42)
> > Byron and Michael, please look at my changeset above and see if that's
> > something we need to add, or is that guaranteed in some other manner?  (I'm
> > tempted to land it anyways as a safety-valve to ensure no stale pointers in
> > case there's a change in invariants.)
> 
> Well, the first hunk in there might be ok, although the second hunk is
> modifying |sctp_it_ctl.cur_it| without acquiring the SCTP_ITERATOR_LOCK.

Looking deeper at how cur_it is used, the 'it' in cur_it can never be in the iteratorhead list (see sctputil.c), so these patches are not needed (you could replace them with asserts that this is the case, however).  Since the sctp crash reported here has been fixed, I don't think we need these.  Upstream should consider adding assert()s to ensure no future breaking of the prerequisites here.

Comment 44

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #38)
> > To make sure that we run the test case in the same way, I just uploaded the
> > node.js servers I used in my testing. Basically the same steps as usual.
> 
> Thanks, that may help a lot in reproducing as this seems rather timing
> dependent.

Have you guys tried to reproduce it again?

Comment 45

[Andrew McCreight [:mccr8]]

Randell, have you had a chance to look at this?

Comment 46

[Andrew McCreight [:mccr8]]

Maire, is there somebody who can look into trying to repro this? Thanks.

Comment 47

[Randell Jesup [:jesup] (needinfo me)]

I've tried again both on win10 (Nightly 53) and linux (local opt inbound build) talking to a node server on the linux box (on the same LAN).  No problems; let it loop for >15 min.  I wasn't running it on the Windows machine itself (don't have node there).  Nils and I (and dveditz) failed to repro before.

Looben: sorry to bother you again, but can you retry with a current Nightly build?

Perhaps it's perf related, if Looben can still repro.  If he can't maybe it really it was bug 1293347... though he tried with the patch and it didn't help, so probably not.

Looben: also if it repros for you, do you have a linux box you can try it on, and if so, can you try an ASAN build off m-c?

Thanks

Comment 48

[Maire Reavy [:mreavy]]

(In reply to Andrew McCreight [:mccr8] from comment #46)
> Maire, is there somebody who can look into trying to repro this? Thanks.

As Jesup said, both he and Nils (also on the WebRTC team) have been trying to repro with no success, and dveditz failed as well. If we can get a handle on reproducing it, Jesup is ready to take this, but right now, we don't have any breadcrumbs to follow.  

Looben:  If you can help us with reproducing this and additional info, we'd really appreciate it!

Comment 49

[Looben Yang]

Seems it's not reproducible with latest nightly: 53.0a1 (2017-01-11) (32-bit).
I've tried several times yesterday but could not reproduce it with this build.
Probably it's been fixed by some other code changes somehow.

Comment 50

[Randell Jesup [:jesup] (needinfo me)]

Ok, thanks.   I still feel like this is most likely bug 1293347.

Looben, if it won't reproduce for you in a build from 
http://archive.mozilla.org/pub/firefox/nightly/2016/09/2016-09-01-03-02-02-mozilla-central/
and does repro in
http://archive.mozilla.org/pub/firefox/nightly/2016/08/2016-08-16-03-04-59-mozilla-central/
then I think we close it as a duplicate.  (We could narrow down to a specific day, but that covers the time period where the fix above merged to mozilla-central.)

If you don't have time to try that, please let me know.  Thanks for all you've done.

Comment 51

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #50)
> Ok, thanks.   I still feel like this is most likely bug 1293347.
> 
> Looben, if it won't reproduce for you in a build from 
> http://archive.mozilla.org/pub/firefox/nightly/2016/09/2016-09-01-03-02-02-
> mozilla-central/
> and does repro in
> http://archive.mozilla.org/pub/firefox/nightly/2016/08/2016-08-16-03-04-59-
> mozilla-central/
> then I think we close it as a duplicate.  (We could narrow down to a
> specific day, but that covers the time period where the fix above merged to
> mozilla-central.)
> 
> If you don't have time to try that, please let me know.  Thanks for all
> you've done.

I have no access to bug 1293347.

It has been reproduced with builds newer then 2016-09-01-03-02-02-mozilla-central/, records were on this report:


"OS: Windows 10
Firefox version: 51.0a1 (2016-09-05)"

"

Reproduced with this second try build too:

51.0a1 (2016-09-17) (32-bit)
"

Comment 52

[Randell Jesup [:jesup] (needinfo me)]

thanks.  If it (still) reproduces in those builds, then it can't be the bug I cited.

can you try current Beta (51), and Aurora/Developer Edition (52)?  If it repros in one of those, then probably there's a fix worth uplifting somewhere we need to track down, or something is masking the bug in 53.

Hopefully it's not some weird compiler/etc issue.

Comment 53

[Randell Jesup [:jesup] (needinfo me)]

Didn't NI reporter with comment 52

Comment 54

[Looben Yang]

(In reply to Randell Jesup [:jesup] from comment #52)
> thanks.  If it (still) reproduces in those builds, then it can't be the bug
> I cited.
> 
> can you try current Beta (51), and Aurora/Developer Edition (52)?  If it
> repros in one of those, then probably there's a fix worth uplifting
> somewhere we need to track down, or something is masking the bug in 53.
> 
> Hopefully it's not some weird compiler/etc issue.

Just tried, but failed to reproduce it on beta: 52.0b1 (32-bit)

Comment 55

[Randell Jesup [:jesup] (needinfo me)]

Ok.  I still think it most likely was bug 1293347, but in any case it's gone now.  I presume it also can't be reproed in 51.  Thanks so much for all the re-testing, given we've been unable to repro here (with many attempts.)

Comment 56

[Daniel Veditz [:dveditz]]

Looben: I've CC'd you on bug 1293347. We think it's a duplicate of that and can't reproduce your results. If you find a way to reproduce (and I agree it's suspicious that your nightly crashed after we landed the fix) we can reopen, but in the meantime we can't award a bounty for this one.