Bugzilla

Bug 1794964 involves a third-party inject DLL called `aswJsFlt64.dll`. While analyzing the minidumps from the crash reports associated with that bug, I noticed that the stack contains more function return addresses than [what was showing in our crash infrastructure](https://crash-stats.mozilla.org/signature/?signature=RtlpFreeHeapInternal%20%7C%20RtlFreeHeap%20%7C%20aswjsflt64.dll.t01). With a correct stack unwinding, there should have been additional return addresses pointing to functions in `aswJsFlt64.dll`, but only one was showing. It seems that when we reach a function that lives in that third-party injected DLL, we cannot continue unwinding the stack.

I am not sure if the source of the problem is (but I think it is the second option):
- we do the stack unwinding on the user's machine, and the DLL is not found because by it is listed as `aswjsflt64.dll.t01` in the module list whereas it is `aswjsflt64.dll` on disk;
- or we do the stack unwinding on the server side, so we don't have the DLL at hand when we do stack unwinding.

Proper stack unwinding on Windows requires [unwind data](https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170) that is stored in the DLL itself in the [data directories](https://learn.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-image_data_directory). Crash analysis itself is also severly limited if we don't have all the DLLs involved in the call stack at hand.

I wonder if it may be necessary to collect some third-party DLLs under some circumstances in order to fix this problem. In that case, what do you think the circumstances should be and what could that look like?

Or, alternatively, maybe we could unwind the stack on client side - where all DLLs are available - and send that together with the crash report?

Thanks!

Bug 1794964 involves a third-party injected DLL called `aswJsFlt64.dll`. While analyzing the minidumps from the crash reports associated with that bug, I noticed that the stack contains more function return addresses than [what was showing in our crash infrastructure](https://crash-stats.mozilla.org/signature/?signature=RtlpFreeHeapInternal%20%7C%20RtlFreeHeap%20%7C%20aswjsflt64.dll.t01). With a correct stack unwinding, there should have been additional return addresses pointing to functions in `aswJsFlt64.dll`, but only one was showing. It seems that when we reach a function that lives in that third-party injected DLL, we cannot continue unwinding the stack.

I am not sure if the source of the problem is (but I think it is the second option):
- we do the stack unwinding on the user's machine, and the DLL is not found because by it is listed as `aswjsflt64.dll.t01` in the module list whereas it is `aswjsflt64.dll` on disk;
- or we do the stack unwinding on the server side, so we don't have the DLL at hand when we do stack unwinding.

Proper stack unwinding on Windows requires [unwind data](https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170) that is stored in the DLL itself in the [data directories](https://learn.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-image_data_directory). Crash analysis itself is also severly limited if we don't have all the DLLs involved in the call stack at hand.

I wonder if it may be necessary to collect some third-party DLLs under some circumstances in order to fix this problem. In that case, what do you think the circumstances should be and what could that look like?

Or, alternatively, maybe we could unwind the stack on client side - where all DLLs are available - and send that together with the crash report?

Thanks!

Bug 1794064 involves a third-party injected DLL called `aswJsFlt64.dll`. While analyzing the minidumps from the crash reports associated with that bug, I noticed that the stack contains more function return addresses than [what was showing in our crash infrastructure](https://crash-stats.mozilla.org/signature/?signature=RtlpFreeHeapInternal%20%7C%20RtlFreeHeap%20%7C%20aswjsflt64.dll.t01). With a correct stack unwinding, there should have been additional return addresses pointing to functions in `aswJsFlt64.dll`, but only one was showing. It seems that when we reach a function that lives in that third-party injected DLL, we cannot continue unwinding the stack.

I am not sure if the source of the problem is (but I think it is the second option):
- we do the stack unwinding on the user's machine, and the DLL is not found because by it is listed as `aswjsflt64.dll.t01` in the module list whereas it is `aswjsflt64.dll` on disk;
- or we do the stack unwinding on the server side, so we don't have the DLL at hand when we do stack unwinding.

Proper stack unwinding on Windows requires [unwind data](https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170) that is stored in the DLL itself in the [data directories](https://learn.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-image_data_directory). Crash analysis itself is also severly limited if we don't have all the DLLs involved in the call stack at hand.

I wonder if it may be necessary to collect some third-party DLLs under some circumstances in order to fix this problem. In that case, what do you think the circumstances should be and what could that look like?

Or, alternatively, maybe we could unwind the stack on client side - where all DLLs are available - and send that together with the crash report?

Thanks!

Bug 1794064 involves a third-party injected DLL called `aswJsFlt64.dll`. While analyzing the minidumps from the crash reports associated with that bug, I noticed that the stack contains more function return addresses than [what was showing in our crash infrastructure](https://crash-stats.mozilla.org/signature/?signature=RtlpFreeHeapInternal%20%7C%20RtlFreeHeap%20%7C%20aswjsflt64.dll.t01). With a correct stack unwinding, there should have been additional return addresses pointing to functions in `aswJsFlt64.dll`, but only one was showing. It seems that when we reach a function that lives in that third-party injected DLL, we cannot continue unwinding the stack.

I am not sure if the source of the problem is (but I think it is the second option):
- we do the stack unwinding on the user's machine, and the DLL is not found because it is listed as `aswjsflt64.dll.t01` in the module list whereas it is `aswjsflt64.dll` on disk;
- or we do the stack unwinding on the server side, so we don't have the DLL at hand when we do stack unwinding.

Proper stack unwinding on Windows requires [unwind data](https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170) that is stored in the DLL itself in the [data directories](https://learn.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-image_data_directory). Crash analysis itself is also severly limited if we don't have all the DLLs involved in the call stack at hand.

I wonder if it may be necessary to collect some third-party DLLs under some circumstances in order to fix this problem. In that case, what do you think the circumstances should be and what could that look like?

Or, alternatively, maybe we could unwind the stack on client side - where all DLLs are available - and send that together with the crash report?

Thanks!