Bug 1555123 Comment 0 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

There have been a few bugs recently around content crashes not reporting stack traces (see blocks list). I figured I'd file a parent bug to collect them all in one place. There's also a similarly titled bug in the "see also" list, but there hasn't been any activity in 2 years, so not sure the two causes were related.

In bug 1539449 I modified mozharness such that the string "A content process crashed" causes the task to turn orange no matter what (even if a stack trace is missing). This should at least stop us from accidentally letting content crashes slip through CI. But we still need to figure out why the stacks are missing.

Here are some STR on try:

> $ hg up 5edbe9b1b822
$ curl https://bug1539449.bmoattachments.org/attachment.cgi?id=9065272 | hg import -
$ ./mach try fuzzy -q "win64asan mochimedia !spi"

Also from bug 1545856 comment 4:

> I'm assuming that the child process has crashed before we set up the crash reporter host. In that case the property bag received by the ipc:content-shutdown notification will contain the "abnormal" field but not the "dumpID" one because no minidump has been generated. That would also explain why there's no stack trace and why there's that failure in reftest.jsm. That should certainly be fixed.
>
> There's also another unrelated problem: I don't know exactly how reftests work but there's been issues with other test harnesses in the past because they didn't wait for the crash manager to do its job before removing the expected crash dumps. That would usually lead to hard-to-diagnose races. I'm pretty sure this is not the case here but Andrew mentioned other cases where he's not getting backtraces in comment 1 and that might be because the minidump has been wiped out too early by the test harness.

Is there some kind of event test harnesses should be waiting for before we start tests?
There have been a few bugs recently around content crashes not reporting stack traces (see blocks list). I figured I'd file a parent bug to collect them all in one place. There's also a similarly titled bug in the "see also" list, but there hasn't been any activity in 2 years, so not sure the two causes were related.

In bug 1539449 I modified mozharness such that the string "A content process crashed" causes the task to turn orange no matter what (even if a stack trace is missing). This should at least stop us from accidentally letting content crashes slip through CI. But we still need to figure out why the stacks are missing.

Here are some STR on try:

```shell
$ hg up 5edbe9b1b822
$ curl https://bug1539449.bmoattachments.org/attachment.cgi?id=9065272 | hg import -
$ ./mach try fuzzy -q "win64asan mochimedia !spi"
```

Also from bug 1545856 comment 4:

> I'm assuming that the child process has crashed before we set up the crash reporter host. In that case the property bag received by the ipc:content-shutdown notification will contain the "abnormal" field but not the "dumpID" one because no minidump has been generated. That would also explain why there's no stack trace and why there's that failure in reftest.jsm. That should certainly be fixed.
>
> There's also another unrelated problem: I don't know exactly how reftests work but there's been issues with other test harnesses in the past because they didn't wait for the crash manager to do its job before removing the expected crash dumps. That would usually lead to hard-to-diagnose races. I'm pretty sure this is not the case here but Andrew mentioned other cases where he's not getting backtraces in comment 1 and that might be because the minidump has been wiped out too early by the test harness.

Is there some kind of event test harnesses should be waiting for before we start tests?
There have been a few bugs recently around content crashes not reporting stack traces (see blocks list). I figured I'd file a parent bug to collect them all in one place. There's also a similarly titled bug in the "see also" list, but there hasn't been any activity in 2 years, so not sure the two causes were related.

In bug 1539449 I modified mozharness such that the string "A content process crashed" causes the task to turn orange no matter what (even if a stack trace is missing). This should at least stop us from accidentally letting content crashes slip through CI. But we still need to figure out why the stacks are missing.

Here are some STR on try:

```shell
$ hg up 5edbe9b1b822
$ curl https://bug1539449.bmoattachments.org/attachment.cgi?id=9065272 | hg import -
$ ./mach try fuzzy -q "win64asan mochimedia !spi"
```

Also from bug 1545856 comment 4:

> I'm assuming that the child process has crashed before we set up the crash reporter host. In that case the property bag received by the ipc:content-shutdown notification will contain the "abnormal" field but not the "dumpID" one because no minidump has been generated. That would also explain why there's no stack trace and why there's that failure in reftest.jsm. That should certainly be fixed.
>
> There's also another unrelated problem: I don't know exactly how reftests work but there's been issues with other test harnesses in the past because they didn't wait for the crash manager to do its job before removing the expected crash dumps. That would usually lead to hard-to-diagnose races. I'm pretty sure this is not the case here but Andrew mentioned other cases where he's not getting backtraces in comment 1 and that might be because the minidump has been wiped out too early by the test harness.

Is there some kind of event test harnesses should be waiting for before we start tests? Since this behaviour has been observed in reftest, mochitest and wpt, I'd tend to rule out issues related to the test-harness prematurely cleaning up minidumps.

Back to Bug 1555123 Comment 0