1926680 - [ASAN] certutil.exe segfaults on windows11 24H2

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P2)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

we have fixed all the simple obvious things, now we are getting a segfault when running certutil.exe. After downloading items and trying to run the binaries locally, firefox.exe segfaults as well. It appears that all the .exe files I find segfault, this indicates something generic with the libraries we compile with or link with are causing problems.

I am not an expert in ASAN, but I believe it uses some shadow memory map, I don't know if we need to update tools, link with additional libraries, or add different compiler/linker flags.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:glandium- do you have suggestions on how I could move forward here? this is reproducible, maybe you know something about the compiler/linker chains or some debugging tip I could do since I can reproduce this.

Comment 2

[Mike Hommey [:glandium]]

Yannis, could you check this out?

Comment 3

[Yannis Juglaret [:yannis]]

Hi, I'm happy to help! We are talking about C:\Windows\System32\certutil.exe right? What command line do you run to get a segfault? For firefox.exe, can you point to an ASAN build of Firefox corresponding to the current progress on this track? Thanks!

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

certutil.exe from our build artifacts used in testing. Here is a try push:
https://treeherder.mozilla.org/jobs?repo=try&searchStr=asan&revision=e8a35a9ded7ca55653c125045eb7e17949753285

you can see all the failures and download artifacts from the build, such as:

• target.installer.exe
• target.common.tar.gz - unpack this and there are tools in there (as well as test harness files and some tests)
• target.zip - instead of the .exe installer

Comment 5

[Yannis Juglaret [:yannis]]

(In reply to Joel Maher ( :jmaher ) (UTC -8) from comment #4)

certutil.exe from our build artifacts used in testing

Thank you! I did not know about this one -- this makes a lot more sense. The issue is a subtle consequence of the addition of calls to memset in RtlDispatchException in ntdll.dll 10.0.26100.2161 (edit: in fact, in all Windows 11 24H2 builds so far -- thanks Gov Maharaj for figuring this out). I have filed an issue on github.

Comment 6

[Yannis Juglaret [:yannis]]

Attachment: Bug 1926680 - Add support for Win11 24H2 in ASan builds. r=#firefox-build-system-reviewers

This patch adds a temporary workaround when building compiler-rt to
ensure that we can run ASan builds on Windows 11 24H2. The proper fix
will come through llvm-project issue #114793 on github.

Comment 7

[Yannis Juglaret [:yannis]]

The attached patch can be used to run ASAN builds on Windows 11 24H2. Discussion whether we want to land it or wait for the proper fix will happen in the code review.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7656bdae4fdc
Add support for Win11 24H2 in ASan builds. r=firefox-build-system-reviewers,glandium

Comment 9

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/7656bdae4fdc