1883067 - Crash in [@ LdrpSnapModule] - Crash with G DATA Software AG's ExploitProtection64.dll

Status: RESOLVED FIXED

(External Software Affecting Firefox :: Other, defect, P3)

Description

[Yannis Juglaret [:yannis]]

In this crash signature, we seem to be mostly crashing with the third-party DLL ExploitProtection64.dll being present in memory, while we are delay-loading another DLL (like in the example crash stack). So I will focus on these crashes here.

Example crash: here.

Top stack frames:

 # Child-SP          RetAddr               Call Site
00 0000005c`2aabbf20 00007ffc`b21310a3     ntdll!LdrpSnapModule+0x222
01 0000005c`2aabc070 00007ffc`b21316c4     ntdll!LdrpProcessWork+0x43
02 0000005c`2aabc0c0 00007ffc`b211baed     ntdll!LdrpDrainWorkQueue+0x184
03 0000005c`2aabc100 00007ffc`b2123694     ntdll!LdrpLoadDllInternal+0x1ad
04 0000005c`2aabc1a0 00007ffc`b21152de     ntdll!LdrpLoadForwardedDll+0x140
05 0000005c`2aabc4c0 00007ffc`b211c007     ntdll!LdrpGetDelayloadExportDll+0xa2
06 0000005c`2aabc5d0 00007ffc`b21160ae     ntdll!LdrpHandleProtectedDelayload+0x87
07 0000005c`2aabcbb0 00007ffc`adfea682     ntdll!LdrResolveDelayLoadedAPI+0xbe
08 0000005c`2aabcc30 00007ffc`adff46b1     dnsapi!_delayLoadHelper2+0x32
09 0000005c`2aabcc70 00007ffc`adfe88a4     dnsapi!_tailMerge_iphlpapi_dll+0x3f
0a 0000005c`2aabcce0 00007ffc`adfe8709     dnsapi!DnsApiInit+0x17c
0b 0000005c`2aabcd10 00007ffc`adff558b     dnsapi!startInit+0x41
0c 0000005c`2aabcd40 00007ffc`adff40ea     dnsapi!DllMain+0x1df
0d 0000005c`2aabcd70 00007ffc`b211ff57     dnsapi!_DllMainCRTStartup+0x76
0e 0000005c`2aabcde0 00007ffc`b215308e     ntdll!LdrpCallInitRoutine+0x6b
0f 0000005c`2aabce50 00007ffc`b2152e3e     ntdll!LdrpInitializeNode+0x1ca
10 0000005c`2aabcf90 00007ffc`b212db8b     ntdll!LdrpInitializeGraphRecurse+0x42
11 0000005c`2aabcfd0 00007ffc`b211bb4f     ntdll!LdrpPrepareModuleForExecution+0xbf
12 0000005c`2aabd010 00007ffc`b2123694     ntdll!LdrpLoadDllInternal+0x20f
13 0000005c`2aabd0b0 00007ffc`b21152de     ntdll!LdrpLoadForwardedDll+0x140
14 0000005c`2aabd3d0 00007ffc`b211c007     ntdll!LdrpGetDelayloadExportDll+0xa2
15 0000005c`2aabd4e0 00007ffc`b21160ae     ntdll!LdrpHandleProtectedDelayload+0x87
16 0000005c`2aabdac0 00007ffc`aeb8fb42     ntdll!LdrResolveDelayLoadedAPI+0xbe
17 0000005c`2aabdb40 00007ffc`aeb91590     mswsock!_delayLoadHelper2+0x32
18 0000005c`2aabdb80 00007ffc`aeb8ddb8     mswsock!_tailMerge_dnsapi_dll+0x3f
19 0000005c`2aabdbf0 00007ffc`aeb8dec8     mswsock!Dns_NSPStartup+0x54
1a 0000005c`2aabdc20 00007ffc`b1d065b3     mswsock!NSPStartup+0x28
1b 0000005c`2aabdc50 00007ffc`b1d06745     ws2_32!NSPROVIDER::Initialize+0xef
1c 0000005c`2aabdf50 00007ffc`b1d06272     ws2_32!NSCATALOG::LoadProvider+0xb9
1d 0000005c`2aabdf80 00007ffc`b1d0b8e5     ws2_32!LookupBeginEnumerationProc+0xda
1e 0000005c`2aabdfc0 00007ffc`b1d0b773     ws2_32!NSQUERY::LookupServiceBegin+0xe5
1f 0000005c`2aabe080 00007ffc`b1d2ecb8     ws2_32!WSALookupServiceBeginW+0x113
20 0000005c`2aabe0e0 00007ffc`b1d16f72     ws2_32!WSALookupServiceBeginA+0x98
21 0000005c`2aabe130 00007ffc`b1d28988     ws2_32!getxyDataEnt+0x84
22 0000005c`2aabe190 00007ffc`037c8801     ws2_32!gethostname+0x128
23 0000005c`2aabe3a0 00007ffc`037abcc5     xul!mozilla::net::nsSocketTransportService::Run+0x81 [/builds/worker/checkouts/gecko/netwerk/base/nsSocketTransportService2.cpp @ 1079] 

We are crashing in Microsoft-internal code (LdrpSnapModule) while it is parsing ntdll's IMAGE_EXPORT_DIRECTORY (presumably to resolve the imports of the delay-loaded DLL). This is what the parsing code around the crash site looks like:

mov     r8d,dword ptr [rbx+1Ch] // read IMAGE_EXPORT_DIRECTORY.AddressOfFunctions
add     r8,r15
mov     qword ptr [rsp+98h],r8
mov     edx,dword ptr [rbx+18h] // read IMAGE_EXPORT_DIRECTORY.NumberOfNames
mov     dword ptr [rsp+70h],edx
mov     ecx,dword ptr [rbx+20h] // read IMAGE_EXPORT_DIRECTORY.AddressOfNames (--> crash!)

During this parsing, rbx points to the base address of ntdll plus the RVA for the IMAGE_EXPORT_DIRECTORY read from ntdll in memory. It is not the original export directory though, because rbx points outside of ntdll. Yet, through rbx, we read the correct value for NumberOfNames (i.e. the same value as the one found on disk for the same version of ntdll), which proves that rbx is intentionally pointing to a copy of the original export directory; it is not pointing to random data (in particular, no bitflip is involved here). In summary, the export directory of ntdll has been hijacked, presumably by G DATA Software AG, and rbx points to the new export directory (very similar to what our ExportDirectoryPatcher test code does).

We manage to read AddressOfFunctions and NumberOfNames because these values still fall within mapped memory, however AddressOfNames falls at the start of the next memory page, which seems committed but with PAGE_NO_ACCESS:

0:009> r rbx
rbx=00007ffcb49fefe0
0:009> ? rbx + 0x20
Evaluate expression: 140723338866688 = 00007ffc`b49ff000
0:009> !address

// pages for ntdll
        BaseAddress      EndAddress+1        RegionSize     Type       State                 Protect             Usage
--------------------------------------------------------------------------------------------------------------------------
+     7ffc`b209d000     7ffc`b20e0000        0`00043000             MEM_FREE    PAGE_NOACCESS                      Free
+     7ffc`b20e0000     7ffc`b20e1000        0`00001000 MEM_IMAGE   MEM_COMMIT  PAGE_READONLY                      Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
      7ffc`b20e1000     7ffc`b220c000        0`0012b000 MEM_IMAGE   MEM_COMMIT  PAGE_EXECUTE_READ                  Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
      7ffc`b220c000     7ffc`b2254000        0`00048000 MEM_IMAGE   MEM_COMMIT  PAGE_READONLY                      Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
      7ffc`b2254000     7ffc`b2255000        0`00001000 MEM_IMAGE   MEM_COMMIT  PAGE_READWRITE                     Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
      7ffc`b2255000     7ffc`b2257000        0`00002000 MEM_IMAGE   MEM_COMMIT  PAGE_WRITECOPY                     Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
      7ffc`b2257000     7ffc`b2260000        0`00009000 MEM_IMAGE   MEM_COMMIT  PAGE_READWRITE                     Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
      7ffc`b2260000     7ffc`b22e9000        0`00089000 MEM_IMAGE   MEM_COMMIT  PAGE_READONLY                      Image      [ntdll; "C:\Windows\System32\ntdll.dll"]
+     7ffc`b22e9000     7ffc`b2630000        0`00347000             MEM_FREE    PAGE_NOACCESS                      Free

// pages for the new export directory
        BaseAddress      EndAddress+1        RegionSize     Type       State                 Protect             Usage
--------------------------------------------------------------------------------------------------------------------------
      7ffc`b49fe000     7ffc`b49ff000        0`00001000 MEM_PRIVATE MEM_COMMIT  PAGE_READWRITE                     <unknown>
      7ffc`b49ff000     7ffc`b4a00000        0`00001000 MEM_PRIVATE MEM_COMMIT  PAGE_NOACCESS                      <unknown>

I will contact the vendor and share this information.

Comment 1

[Erik Schulz]

Hi, I am a developer working for G DATA on the exploit protection module.

We had a look at your analysis of the issue and can confirm that the crashes of Firefox are most likely caused by this protection module, though we are still searching for the underlying root cause.
I’ve tried reproducing this crash with the same version of Firefox and our ExploitProtection as in your analysis, but so far I’ve been unable to do so.

Still, I’ve just disabled this particular protection method for Firefox via a signature update until we find the root cause, meaning that from now on we won’t inject ExploitProtection64.dll into Firefox anymore.
This should fix all of these crashes on your side and is hopefully already enough for you to close this bugticket. We would reopen the ticket or create a new one, if we need any kind of help by the Mozilla team to solve the root cause once we find it.
Note that most of our customers should update within hours, but there is always a small percentage of customers that do not update their software, so we do not expect this issue to disappear completely from your crash telemetry.

Comment 2

[Gian-Carlo Pascutto [:gcp]]

Crash data pretty much matches what the developer predicted. I don't think we need further action.