1417772 - Perma Tier 2 Central TestDllInterceptor.exe | test failed with return code 2147483651 after Assertion failure: false (MOZ_ASSERT_UNREACHABLE: Unrecognized opcode sequence)

Reporter

Description

•

7 years ago

treeherder

Filed by: csabou [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=145154648&repo=mozilla-central

https://queue.taskcluster.net/v1/task/Aue8UB7BRx67nAPkuh0_4A/runs/0/artifacts/public/logs/live_backing.log

23:47:37     INFO -  TEST-START | TestDllInterceptor.exe
23:47:37     INFO -  PID 6732 |
23:47:37     INFO -  Assertion failure: false (MOZ_ASSERT_UNREACHABLE: Unrecognized opcode sequence), at z:/build/build/src/obj-firefox/dist/include\nsWindowsDllInterceptor.h:895
23:47:37  WARNING -  TEST-UNEXPECTED-FAIL | TestDllInterceptor.exe | test failed with return code 2147483651
23:47:37     INFO -  TEST-INFO took 423ms

Comment hidden (Intermittent Failures Robot)

16 failures in 762 pushes (0.021 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 15
* try: 1

Platform breakdown:
* windows10-64-ccov: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1417772&startday=2017-11-13&endday=2017-11-19&tree=all

Firefox Bug Husbandry Bot

Comment 2

•

6 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Status: NEW → RESOLVED

Closed: 6 years ago

Resolution: --- → INCOMPLETE

Cristina Coroiu [:ccoroiu]

Comment 3

•

6 years ago

Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=185162586&repo=autoland&lineNumber=988

13:41:12     INFO -  TEST-START | TestCheckedInt.exe
13:41:12     INFO -  PID 1828 |
13:41:12     INFO -  0 tests failed, 5066 tests passed out of 5066 tests, covering 11 distinct integer types.
13:41:12     INFO -  TEST-PASS | TestCheckedInt.exe | took 176ms
13:41:12     INFO -  TEST-START | TestCountPopulation.exe
13:41:12     INFO -  TEST-PASS | TestCountPopulation.exe | took 174ms
13:41:12     INFO -  TEST-START | TestCountZeroes.exe
13:41:12     INFO -  TEST-PASS | TestCountZeroes.exe | took 122ms
13:41:12     INFO -  TEST-START | TestDefineEnum.exe
13:41:12     INFO -  TEST-PASS | TestDefineEnum.exe | took 171ms
13:41:12     INFO -  TEST-START | TestDllInterceptor.exe
13:41:12  WARNING -  TEST-UNEXPECTED-FAIL | TestDllInterceptor.exe | test failed with return code 3221225794
13:41:12     INFO -  TEST-INFO took 170ms
13:41:12     INFO -  TEST-START | TestEndian.exe
13:41:13     INFO -  TEST-PASS | TestEndian.exe | took 121ms
13:41:13     INFO -  TEST-START | TestEnumeratedArray.exe
13:41:13     INFO -  TEST-PASS | TestEnumeratedArray.exe | took 171ms
13:41:13     INFO -  TEST-START | TestEnumSet.exe
13:41:13     INFO -  TEST-PASS | TestEnumSet.exe | took 171ms
13:41:13     INFO -  TEST-START | TestEnumTypeTraits.exe
13:41:13     INFO -  TEST-PASS | TestEnumTypeTraits.exe | took 169ms
13:41:13     INFO -  TEST-START | TestFastBernoulliTrial.exe
13:41:13     INFO -  TEST-PASS | TestFastBernoulliTrial.exe | took 176ms
13:41:13     INFO -  TEST-START | TestFloatingPoint.exe
13:41:13     INFO -  TEST-PASS | TestFloatingPoint.exe | took 172ms
13:41:13     INFO -  TEST-START | TestIntegerPrintfMacros.exe
13:41:14     INFO -  TEST-PASS | TestIntegerPrintfMacros.exe | took 125ms
13:41:14     INFO -  TEST-START | TestIntegerRange.exe
13:41:14     INFO -  TEST-PASS | TestIntegerRange.exe | took 174ms

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

1 failures in 718 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1417772&startday=2018-06-25&endday=2018-07-01&tree=trunk

Firefox Bug Husbandry Bot

Comment 5

•

6 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Status: REOPENED → RESOLVED

Closed: 6 years ago → 6 years ago

Resolution: --- → INCOMPLETE

Razvan Maries

Comment 6

•

5 years ago

New occurrences: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=215900517&repo=mozilla-central&lineNumber=1114

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

1 failures in 1862 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1417772&startday=2018-12-03&endday=2018-12-09&tree=all

Firefox Bug Husbandry Bot

Comment 8

•

5 years ago

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Status: REOPENED → RESOLVED

Closed: 6 years ago → 5 years ago

Resolution: --- → INCOMPLETE

Noemi Erli[:noemi_erli]

Comment 9

•

5 years ago

Recent log failure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=242990385&repo=autoland&lineNumber=1045

20:56:06 INFO - TEST-START | TestDllInterceptor.exe
20:56:06 INFO - PID 7076 |
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Hook added
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Hook called
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Hook works properly
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Hook was not called after unregistration
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Original function worked properly
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook GetWindowInfo from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function GetWindowInfo from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook TrackPopupMenu from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function TrackPopupMenu from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook QueryDosDeviceW from kernelbase.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function QueryDosDeviceW from kernelbase.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could detour CreateWindowExW from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function CreateWindowExW from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InSendMessageEx from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InSendMessageEx from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook ImmGetCompositionStringW from imm32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function ImmGetCompositionStringW from imm32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook ImmSetCandidateWindow from imm32.dll
20:56:06 INFO - TEST-SKIPPED | WindowsDllInterceptor | Will not attempt to execute patched ImmSetCandidateWindow.
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook ImmNotifyIME from imm32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function ImmNotifyIME from imm32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook GetKeyState from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function GetKeyState from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook LdrUnloadDll from ntdll.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function LdrUnloadDll from ntdll.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook LdrResolveDelayLoadedAPI from ntdll.dll
20:56:06 INFO - TEST-SKIPPED | WindowsDllInterceptor | Will not attempt to execute patched LdrResolveDelayLoadedAPI.
20:56:06 INFO - TEST-SKIPPED | WindowsDllInterceptor | Skipped hook test for RtlInstallFunctionTableCallback from kernel32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook ProcessCaretEvents from tiptsf.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function ProcessCaretEvents from tiptsf.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook SetCursorPos from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function SetCursorPos from user32.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetOpenA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetOpenA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetCloseHandle from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetCloseHandle from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetConnectA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetConnectA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetQueryDataAvailable from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetQueryDataAvailable from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetReadFile from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetReadFile from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetWriteFile from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetWriteFile from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook InternetSetOptionA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function InternetSetOptionA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Could hook HttpAddRequestHeadersA from wininet.dll
20:56:06 INFO - TEST-PASS | WindowsDllInterceptor | Executed hooked function HttpAddRequestHeadersA
20:56:06 WARNING - TEST-UNEXPECTED-FAIL | TestDllInterceptor.exe | test failed with return code 3221225477
20:56:06 INFO - TEST-INFO took 83ms

Status: RESOLVED → REOPENED

Resolution: INCOMPLETE → ---

Comment hidden (Intermittent Failures Robot)

(No longer employed by Mozilla) Aaron Klotz

Updated

•

5 years ago

Component: General → mozglue

OS: Unspecified → Windows

Priority: P5 → P3

Natalia Csoregi [:nataliaCs]

Updated

•

5 years ago

Summary: Intermittent TestDllInterceptor.exe | test failed with return code 2147483651 → Intermittent TestDllInterceptor.exe | test failed with return code 2147483651 after Assertion failure: false (MOZ_ASSERT_UNREACHABLE: Unrecognized opcode sequence)

Comment hidden (Intermittent Failures Robot)

Bogdan Tara[:bogdan_tara | bogdant]

Comment 16

•

5 years ago

This has started to perma fail on Central, Tier 2.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=windows%2C10%2Cx64%2Cccov%2Cdebug%2Ctest-windows10-64-ccov%2Fdebug-cppunit-1proc%2C%28cppunit%29&fromchange=80ab213de5a9647fb7da0ed969a3c5a0d5ce4cdf&tochange=1469feeca395eff65a81d6f283da42b0866e6717&selectedJob=277708806

Summary: Intermittent TestDllInterceptor.exe | test failed with return code 2147483651 after Assertion failure: false (MOZ_ASSERT_UNREACHABLE: Unrecognized opcode sequence) → Perma Tier 2 Central TestDllInterceptor.exe | test failed with return code 2147483651 after Assertion failure: false (MOZ_ASSERT_UNREACHABLE: Unrecognized opcode sequence)

Bogdan Tara[:bogdan_tara | bogdant]

Comment 17

•

5 years ago

This seems to have started with the changes from bug 1596930.

https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=windows%2C10%2Cx64%2Cccov%2Cdebug%2Ctest-windows10-64-ccov%2Fdebug-cppunit-1proc%2C%28cppunit%29&tochange=ddd503f15330cdbacdb626759e885f79146148c2&fromchange=988611a6b22427477f562a5b8c3134530464ded1&selectedJob=277732094

toshi, could you, please, take a look?

Flags: needinfo?(tkikuchi)

Toshihito Kikuchi [:toshi]

Assignee

Comment 18

•

5 years ago

I think this is caused by a new test code, not product code. If so, adding #ifdef MOZ_CODE_COVERAGE to a right place should be good.

Is there any way to get TestDllInterceptor.exe and its symbol from "Windows 10 x64 CCov debug" build? I need to debug it locally.

Flags: needinfo?(tkikuchi) → needinfo?(btara)

Bogdan Tara[:bogdan_tara | bogdant]

Comment 19

•

5 years ago

toshi, i only know where to download nightly builds, if that helps:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&job_type_symbol=ns%2Cnr&fromchange=1469feeca395eff65a81d6f283da42b0866e6717&tochange=2c912e46295e4f8a3fa5824a7f378e06760ec7dd&selectedJob=277745026

Bogdan Tara[:bogdan_tara | bogdant]

Updated

•

5 years ago

Flags: needinfo?(btara)

Comment hidden (Intermittent Failures Robot)

Toshihito Kikuchi [:toshi]

Assignee

Comment 21

•

5 years ago

I couldn't find pdb symbols, but I could get at least TestDllInterceptor.exe from a job I submitted.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d1bebbb270e8c99001262cacf01b83fcccf0e184&selectedJob=278086453

This test failure is expected because the assembly functions with CCov instrumentation have the patterns we cannot support for now. We need to skip these test cases.

0:000> u TestDllInterceptor!MovPushRet l3
TestDllInterceptor!MovPushRet:
00007ff7`169f1000 488305f840090001 add     qword ptr [TestDllInterceptor!rotatePayload+0x940c0 (00007ff7`16a85100)],1
00007ff7`169f1008 48b800000000ff7f0000 mov rax,7FFF00000000h
00007ff7`169f1012 50              push    rax
0:000> u TestDllInterceptor!MovRaxJump l3
TestDllInterceptor!MovRaxJump:
00007ff7`169f1020 488305e040090001 add     qword ptr [TestDllInterceptor!rotatePayload+0x940c8 (00007ff7`16a85108)],1
00007ff7`169f1028 48b800000000ff7f0000 mov rax,7FFF00000000h
00007ff7`169f1032 ffe0            jmp     rax

Toshihito Kikuchi [:toshi]

Assignee

Updated

•

5 years ago

Assignee: nobody → tkikuchi

Toshihito Kikuchi [:toshi]

Assignee

Comment 22

•

5 years ago

Attached file Bug 1417772 - Skip testcases in TestAssemblyFunctions in CCov build. r=handyman — Details

Our detour cannot handle assembly patterns which is injected by the code coverage
instrumentation. We need to skip them in CCov build.

Pulsebot

Comment 23

•

5 years ago

Pushed by dluca@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7dfa4aaad8f3
Skip testcases in TestAssemblyFunctions in CCov build. r=handyman

Andreea Pavel [:apavel]

Comment 24

•

5 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/7dfa4aaad8f3

Status: REOPENED → RESOLVED

Closed: 5 years ago → 5 years ago

status-firefox72: --- → fixed

Resolution: --- → FIXED

Target Milestone: --- → mozilla72

Comment hidden (Intermittent Failures Robot)