I think I've spent too much time on this and haven't got a working solution, so I I'll put this on pause for now, since a (albeit annoying) workaround does exists. I'll write down what I've tried and what I'm thinking for next steps just to make it easier to continue this in the future. As mentioned previously, I spoke with Glandium and his fix was to just disable ASAN for in Rust. While this might sound easy, I wasn't able to get this to work. To enable ASAN, the mozconfig setting eventually adds `-fsanitize=address` to `CFLAGS`, `CXXFLAGS`, and `LDFLAGS`. Eventually in Mach/Python land, we invoke the root makefile the builds both C/C++ and Rust code, so we can't disable it there, only inside the makefile(s). In [rust.mk](https://searchfox.org/mozilla-central/source/config/makefiles/rust.mk) I tried to `filter-out` the `-fsanitize=address` and re-export the flags, but this didn't work. I used `$(foreach v, $(.VARIABLES), $(if $(filter file,$(origin $(v))), $(info $(v)=$($(v)))))` to get all the env variables visible in the file and the ones I listed weren't visible (which is suspicious), but `-fsanitize=address` was present in `OS_CXXFLAGS` and `COMPUTED_CXX_LDFLAGS`. I removed the `-fsanitize=address` from them, and re-exported them with this code: ``` OS_CXXFLAGS :=$(filter-out -fsanitize=address,$(OS_CXXFLAGS)) export OS_CXXFLAGS:=$(OS_CXXFLAGS) COMPUTED_CXX_LDFLAGS :=$(filter-out -fsanitize=address,$(COMPUTED_CXX_LDFLAGS)) export COMPUTED_CXX_LDFLAGS:=$(COMPUTED_CXX_LDFLAGS) ``` (Eventually this would be wrapped in `ifeq (WINNT,$(HOST_OS_ARCH))` to only do this on Windows, but I left that out for now) But that didn't work, so I tried the same for the `CFLAGS`, `CXXFLAGS`, and `LDFLAGS`, but that also didn't work either. I can't figure out why, since that's essentially the purpose of this file, as per the header comment: ``` # /!\ In this file, we export multiple variables globally via make rather than # in recipes via the `env` command to avoid round-trips to msys on Windows, which # tend to break environment variable values in interesting ways. ``` Glandium also commented that it might be better to fix whatever is wrong in the Rust/Cargo build scripts that's making them crash with ASAN, but that's a bit outside my wheelhouse since I'm not that familiar with Rust/Cargo. Running the build in verbose (eg: `./mach build -v)` gives the command that eventually tries to build the projects that fail with the `STATUS_STACK_BUFFER_OVERRUN` in `build-script-build.exe`, but running it outside of the Mach environment (ie: just through the MSYS shell) doesn't work. It seems as though there are multiple environment settings missing to make this command work. I fixed a few, but got stuck on one I wasn't able to get it to run on its own. Going down this road, it's probably easier to completely isolate the failing projects and try to build them with ASAN independently. For example, grab [serde](https://github.com/serde-rs/serde) and build it outside of our environment with [ASAN enabled](https://clang.llvm.org/docs/AddressSanitizer.html#usage). Though, it's not just `serde`. I also saw `proc-macro2`, `winapi`, and `http3server`. If we continue down this path, we'd need to fix all these third party build scripts and push the patch upstream. I would continue down this path now, but I've exceeded the time limit I've set for myself on this, and there is a viable workaround available. Maybe what I've written can help somebody else more knowledgeable find a solution, but for now I'll leave this on hold. (Though if there is an urgent demand for this I'll definitely jump back in to it).
Bug 1757116 Comment 4 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
I think I've spent too much time on this and haven't got a working solution, so I I'll put this on pause for now, since a (albeit annoying) workaround does exists. I'll write down what I've tried and what I'm thinking for next steps just to make it easier to continue this in the future. As mentioned previously, I spoke with Glandium and his fix was to just disable ASAN for in Rust. While this might sound easy, I wasn't able to get this to work. To enable ASAN, the mozconfig setting eventually adds `-fsanitize=address` to `CFLAGS`, `CXXFLAGS`, and `LDFLAGS`. Eventually in Mach/Python land, we invoke the root makefile the builds both C/C++ and Rust code, so we can't disable it there, only inside the makefile(s). In [rust.mk](https://searchfox.org/mozilla-central/source/config/makefiles/rust.mk) I tried to `filter-out` the `-fsanitize=address` and re-export the flags, but this didn't work. I used `$(foreach v, $(.VARIABLES), $(if $(filter file,$(origin $(v))), $(info $(v)=$($(v)))))` to get all the env variables visible in the file and the ones I listed weren't visible (which is suspicious), but `-fsanitize=address` was present in `OS_CXXFLAGS` and `COMPUTED_CXX_LDFLAGS`. I removed the `-fsanitize=address` from them, and re-exported them with this code: ``` OS_CXXFLAGS :=$(filter-out -fsanitize=address,$(OS_CXXFLAGS)) export OS_CXXFLAGS:=$(OS_CXXFLAGS) COMPUTED_CXX_LDFLAGS :=$(filter-out -fsanitize=address,$(COMPUTED_CXX_LDFLAGS)) export COMPUTED_CXX_LDFLAGS:=$(COMPUTED_CXX_LDFLAGS) ``` (Eventually this would be wrapped in `ifeq (WINNT,$(HOST_OS_ARCH))` to only do this on Windows, but I left that out for now) But that didn't work, so I tried the same for the `CFLAGS`, `CXXFLAGS`, and `LDFLAGS`, but that also didn't work either. I can't figure out why, since that's essentially the purpose of this file, as per the header comment: ``` # /!\ In this file, we export multiple variables globally via make rather than # in recipes via the `env` command to avoid round-trips to msys on Windows, which # tend to break environment variable values in interesting ways. ``` Glandium also mentioned that it might be better to fix whatever is wrong in the Rust/Cargo build scripts that's making them crash with ASAN, but that's a bit outside my wheelhouse since I'm not that familiar with Rust/Cargo. Running the build in verbose (eg: `./mach build -v)` gives the command that eventually tries to build the projects that fail with the `STATUS_STACK_BUFFER_OVERRUN` in `build-script-build.exe`, but running it outside of the Mach environment (ie: just through the MSYS shell) doesn't work. It seems as though there are multiple environment settings missing to make this command work. I fixed a few, but got stuck on one I wasn't able to get it to run on its own. Going down this road, it's probably easier to completely isolate the failing projects and try to build them with ASAN independently. For example, grab [serde](https://github.com/serde-rs/serde) and build it outside of our environment with [ASAN enabled](https://clang.llvm.org/docs/AddressSanitizer.html#usage). Though, it's not just `serde`. I also saw `proc-macro2`, `winapi`, and `http3server`. If we continue down this path, we'd need to fix all these third party build scripts and push the patch upstream. I would continue down this path now, but I've exceeded the time limit I've set for myself on this, and there is a viable workaround available. Maybe what I've written can help somebody else more knowledgeable find a solution, but for now I'll leave this on hold. (Though if there is an urgent demand for this I'll definitely jump back in to it).