1805331 - Perma win ccov TestDllInterceptor.exe | test failed with return code 3221225477

Status: RESOLVED FIXED

(Core :: mozglue, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=399400213&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bZQzg3irT5q0Ra6sChkiCA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-12-12T23:37:27.586Z] 23:37:27     INFO -  TEST-PASS | WindowsDllInterceptor | The hook for FreeCredentialsHandle from sspicli.dll has coherent unwind info.
[task 2022-12-12T23:37:27.586Z] 23:37:27     INFO -  TEST-PASS | WindowsDllInterceptor | Executed hooked function FreeCredentialsHandle from sspicli.dll
[task 2022-12-12T23:37:27.586Z] 23:37:27     INFO -  mozcrash checking C:\Users\task_167088794024593\AppData\Local\Temp\tmpsbou6u06 for minidumps...
[task 2022-12-12T23:37:27.586Z] 23:37:27  WARNING -  TEST-UNEXPECTED-FAIL | TestDllInterceptor.exe | test failed with return code 3221225477
[task 2022-12-12T23:37:27.586Z] 23:37:27     INFO -  TEST-INFO took 109ms
[task 2022-12-12T23:37:27.587Z] 23:37:27     INFO -  TEST-START | TestEndian.exe
[task 2022-12-12T23:37:27.608Z] 23:37:27     INFO -  mozcrash checking C:\Users\task_167088794024593\AppData\Local\Temp\tmpwltwkode for minidumps...
[task 2022-12-12T23:37:27.608Z] 23:37:27     INFO -  TEST-PASS | TestEndian.exe | took 30ms

Comment 1

[Yannis Juglaret [:yannis]]

Oops, looking at this.

Comment 2

[Yannis Juglaret [:yannis]]

Other tests in the same file are guarded to not run under MOZ_CODE_COVERAGE. Let's do the same for the moment. I describe below why the test fails under code coverage instrumentation.

First problem, the code coverage instrumentation modifies the DetouredCallJumper to add the increment of a counter:

TestDllInterceptor!DetouredCallJumper:
00007ff7`e6041310 48ff05698e0500  inc     qword ptr [TestDllInterceptor!payloadNotHooked+0x58e10 (00007ff7`e609a180)]
00007ff7`e6041317 48ff253a8d0500  jmp     qword ptr [TestDllInterceptor!payloadNotHooked+0x58ce8 (00007ff7`e609a058)]

This may prevent the jumper to be recognized by interception code.

Second problem, it also modifies DetouredCallCode despite __attribute__((naked)), which:

• changes the size of DetouredCallCode to something greater than the hardcoded size gDetouredCallCodeSize = 16;
• makes DetouredCallCode's instructions position-dependent because they now increment a value at a given offset from rip.

gDetouredCall is a dynamic copy of DetouredCallCode's bytes, which is assumed to be of size gDetouredCallCodeSize. Because of what's stated above, gDetouredCall now points to code that:

• starts with a increment of an invalid pointer;
• doesn't contain all the instructions from DetouredCallCode.

poi(TestDllInterceptor!gDetouredCall):
000001d3`18b80012 48ff05818e0500  inc     qword ptr [000001d3`18bd8e9a]
000001d3`18b80019 4883ec28        sub     rsp,28h
000001d3`18b8001d 4885c9          test    rcx,rcx
000001d3`18b80020 7402            je      000001d3`18b80024
// Copy stops here
000001d3`18b80022 0000            add     byte ptr [rax],al

As a better solution, I will later try to see if it's possible to tell coverage instrumentation to not modify DetouredCallJumper and DetouredCallCode. If that works we could propagate the fix to other tests in the same file.

Comment 3

[Yannis Juglaret [:yannis]]

Attachment: Bug 1805331 - Disable TestDetouredCallUnwindInfo under MOZ_CODE_COVERAGE in TestDllInterceptor. r=gcp

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8d0768988c02
Disable TestDetouredCallUnwindInfo under MOZ_CODE_COVERAGE in TestDllInterceptor. r=gcp

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1800898

Comment 6

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/8d0768988c02

Comment 7

[Intermittent Failures Robot]

14 failures in 3558 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 7
• autoland: 7

Platform and build breakdown:

• windows10-64-2004-ccov-qr: 14
  • opt: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1805331&startday=2022-12-12&endday=2022-12-18&tree=all