Bugzilla

This crash looks like the consequence of doing an in-process crash dump with an already corrupt heap. Microsoft's code for generating a crash dump itself relies on allocating from the heap, which can fail if the heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place).

In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected, or when we are crashing as the consequence an undetected heap corruption (e.g. with an access violation).

This crash looks like the consequence of doing an in-process crash dump with an already corrupted heap. Microsoft's code for generating a crash dump itself relies on allocating from the heap, which can fail if the heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place).

In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected, or when we are crashing as the consequence an undetected heap corruption (e.g. with an access violation).

This crash looks like the consequence of doing an in-process crash dump with an already corrupted heap. Microsoft's code for generating a crash dump itself relies on allocating from the process' default heap, which can fail if the heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place).

In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected, or when we are crashing as the consequence an undetected heap corruption (e.g. with an access violation).

This crash looks like the consequence of doing an in-process crash dump with an already corrupted process' default heap. Microsoft's code for generating a crash dump itself relies on allocating from the process' default heap, which can fail if that heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place).

In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected, or when we are crashing as the consequence an undetected heap corruption (e.g. with an access violation).

This crash looks like the consequence of doing an in-process crash dump with an already corrupted process' default heap. Microsoft's code for generating a crash dump itself relies on allocating from the process' default heap, which can fail if that heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place).

In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected for that heap, or when we are crashing as the consequence an undetected corruption for that heap (e.g. crashing with an access violation).

This crash looks like the consequence of doing an in-process crash dump with an already corrupted process' default heap. Microsoft's code for generating a crash dump itself relies on allocating from the process' default heap, which can fail if that heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place for these reports).

In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected for that heap, or when we are crashing as the consequence an undetected corruption for that heap (e.g. crashing with an access violation).

This crash looks like the consequence of doing an in-process crash dump with an already corrupted process' default heap. Microsoft's code for generating a crash dump itself relies on allocating from the process' default heap, which can fail if that heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place for these reports).

~~In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected for that heap, or when we are crashing as the consequence an undetected corruption for that heap (e.g. crashing with an access violation).~~

Edited:

Actually, what I wrote doesn't hold. The crashes we have here are main process crashes generated during out-of-process crash generation for a child process. So the situation is rather as follows, i.e. quite the opposite of what I was saying:
- a child process is crashing and requested an out-of-process crash dump;
- the parent process calls into Microsoft code to dump the child process;
- but the parent process' default heap itself is corrupt and this gets detected;
- so we end up doing an in-process crash dump of the parent process and reporting it.

A scenario where this could be happening would be a heap corruption that simultaneously affects multiple processes. For example, some message could be broadcasted by some third-party software to all the clients of its injected DLL, resulting in the same heap corruption in parent and child processes altogether. Then the child process could crash as a result of the heap corruption, and the parent process could have a not-yet-detected heap corruption that we detect while running the code that is supposed to dump the child process.

This crash looks like the consequence of doing an in-process crash dump with an already corrupted process' default heap. Microsoft's code for generating a crash dump itself relies on allocating from the process' default heap, which can fail if that heap is already corrupted (which is the most likely reason why we got in the crash dump code in first place for these reports).

~~In other words, I believe these crashes should be considered as additional reports for other bugs, that provide evidence that the underlying problem is a corruption of the process' default heap on Windows. This can happen either when we are reporting a heap corruption that we just detected for that heap, or when we are crashing as the consequence an undetected corruption for that heap (e.g. crashing with an access violation).~~

Edited:

Actually, what I wrote doesn't hold. The crashes we have here are main process crashes generated during out-of-process crash generation for a child process. So the situation is rather as follows, i.e. quite the opposite of what I was saying:
- a child process is crashing and requested an out-of-process crash dump;
- the parent process calls into Microsoft code to dump the child process;
- but the parent process' default heap itself is corrupt and this gets detected;
- so we end up doing an in-process crash dump of the parent process and reporting it.

It may just be bad luck that we detect the parent heap corruption right at that moment. Or, another scenario where this could be happening would be a heap corruption that simultaneously affects multiple processes. For example, some message could be broadcasted by some third-party software to all the clients of its injected DLL, resulting in the same heap corruption in parent and child processes altogether. Then the child process could crash as a result of the heap corruption, and the parent process could have a not-yet-detected heap corruption that we detect while running the code that is supposed to dump the child process.

(Edited, my previous analysis was wrong)

The crashes we have here are main process crashes generated during out-of-process crash generation for a child process. So the situation is as follows:
- a child process is crashing and requested an out-of-process crash dump;
- the parent process calls into Microsoft code to dump the child process;
- but the parent process' default heap itself is either already corrupt, or gets corrupted by Microsoft's code, and the corruption gets detected;
- so we end up doing an in-process crash dump of the parent process and reporting it.

Like [:gsvelto] mentioned, we have almost exclusively Windows 11 23H2 Insider Preview builds starting with build 10.0.25236. The beginning of the crash spike matches with the release of build 10.0.25236 on November 2nd, 2022. In addition to the possibilities already mentioned by [:gsvelto], I would like to suggest that Microsoft could have added new ways to detect heap corruptions, which could lead a usually undetected heap corruption to be more likely to be detected starting with build 10.0.25236. In that case, the heap corruption we would detect here would not necessarily originate from Microsoft code but could simply have more chance to get caught while we write the crash dump compared to previous builds.

(Edited, my previous analysis was wrong)

The crashes we have here are main process crashes generated during out-of-process crash generation for a child process. So the situation is as follows:
- a child process is crashing and requested an out-of-process crash dump;
- the parent process calls into Microsoft code to dump the child process;
- but the parent process' default heap itself is either already corrupt, or gets corrupted by Microsoft's code, and the corruption gets detected;
- so we end up doing an in-process crash dump of the parent process and reporting it.

Like [:gsvelto] mentioned, we have almost exclusively Windows 11 23H2 Insider Preview builds starting with build 10.0.25236. The beginning of the crash spike matches with the release of build 10.0.25236 on November 2nd, 2022. In addition to the possibilities already mentioned by [:gsvelto], I would like to suggest that Microsoft could have added new ways to detect heap corruptions, which could lead a usually undetected heap corruption to be more likely to be detected starting with build 10.0.25236. In that case, the heap corruption we would detect here would not necessarily originate from Microsoft code, it could get *detected* here but have occurred *before*.

(Edited, my previous analysis was wrong)

The crashes we have here are main process crashes generated during out-of-process crash generation for a child process. So the situation is as follows:
- a child process is crashing and requested an out-of-process crash dump;
- the parent process calls into Microsoft code to dump the child process;
- but the parent process' default heap itself is either already corrupt, or gets corrupted by Microsoft's code, and the corruption gets detected;
- so we end up doing an in-process crash dump of the parent process and reporting it.

Like [:gsvelto] mentioned, we have almost exclusively Windows 11 23H2 Insider Preview builds in the crashes, starting with build 10.0.25236. The beginning of the crash spike matches with the release of build 10.0.25236 on November 2nd, 2022. In addition to the possibilities already mentioned by [:gsvelto], I would like to suggest that Microsoft could have added new ways to detect heap corruptions, which could lead a usually undetected heap corruption to be more likely to be detected starting with build 10.0.25236. In that case, the heap corruption we would detect here would not necessarily originate from Microsoft code, it could get *detected* here but have occurred *before*.