2025119 - Frequent application crashed [[Unknown]] | gfx/tests/crashtests/1652750-deep-scene-stack.html | single tracking bug

Status: RESOLVED FIXED

(Core :: Printing: Output, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: agoloman [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=554940034&repo=autoland&task=RE5Y2AkNSnSU1IzVCFRyHQ.0
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/RE5Y2AkNSnSU1IzVCFRyHQ/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/default/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/RE5Y2AkNSnSU1IzVCFRyHQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2026-03-21T17:34:05.897+00:00] 17:34:05     INFO - REFTEST TEST-PASS | gfx/tests/crashtests/1650990.html | (LOAD ONLY)
[task 2026-03-21T17:34:05.904+00:00] 17:34:05     INFO - REFTEST TEST-END | gfx/tests/crashtests/1650990.html
[task 2026-03-21T17:34:05.910+00:00] 17:34:05     INFO - REFTEST TEST-START | gfx/tests/crashtests/1652750-deep-scene-stack.html
[task 2026-03-21T17:34:05.911+00:00] 17:34:05     INFO - REFTEST TEST-LOAD | file:///D:/task_177411332592375/build/tests/reftest/tests/gfx/tests/crashtests/1652750-deep-scene-stack.html | 1521 / 4178 (36%)
[task 2026-03-21T17:34:07.684+00:00] 17:34:07    ERROR - A content process crashed and MOZ_CRASHREPORTER_SHUTDOWN is set, shutting down
[task 2026-03-21T17:34:07.705+00:00] 17:34:07     INFO - 1774114447704	Marionette	TRACE	Received observer notification quit-application
[task 2026-03-21T17:34:07.706+00:00] 17:34:07     INFO - 1774114447704	Marionette	TRACE	Application is shutting down with reason: "shutdown"
[task 2026-03-21T17:34:07.706+00:00] 17:34:07     INFO - 1774114447704	Marionette	INFO	Stopped listening on port 2828
[task 2026-03-21T17:34:07.745+00:00] 17:34:07     INFO - 1774114447745	Marionette	DEBUG	Marionette stopped listening
[task 2026-03-21T17:34:07.855+00:00] 17:34:07     INFO - !!! error running onStopped callback: TypeError: callback is not a function
[task 2026-03-21T17:34:08.268+00:00] 17:34:08     INFO - [GFX1-]: VideoBridgeParent receives IPC close with reason=AbnormalShutdown
[task 2026-03-21T17:34:08.270+00:00] 17:34:08     INFO - 1774114448269	Marionette	TRACE	Received observer notification xpcom-shutdown
[task 2026-03-21T17:34:08.279+00:00] 17:34:08     INFO - 1774114448278	Marionette	TRACE	Received observer notification xpcom-shutdown-threads
[task 2026-03-21T17:34:08.363+00:00] 17:34:08     INFO - REFTEST INFO | Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/MAsgHmwOTTyad-JQ4T_uRA/artifacts/public/build/target.crashreporter-symbols.zip
[task 2026-03-21T17:34:28.997+00:00] 17:34:28     INFO - REFTEST INFO | Copy/paste: D:/task_177411332592375/fetches\minidump-stackwalk\minidump-stackwalk.exe --symbols-url=https://symbols.mozilla.org/ --cyborg=D:\task_177411332592375\build\blobber_upload_dir\3dfec1fd-9f5f-4469-acc4-db3217f68f8f.json C:\Users\task_177411332592375\AppData\Local\Temp\tmpwy11dunm.mozrunner\minidumps\3dfec1fd-9f5f-4469-acc4-db3217f68f8f.dmp C:\Users\task_177411332592375\AppData\Local\Temp\tmptzc5tslh
[task 2026-03-21T17:34:29.478+00:00] 17:34:29     INFO - REFTEST INFO | Saved minidump as D:\task_177411332592375\build\blobber_upload_dir\3dfec1fd-9f5f-4469-acc4-db3217f68f8f.dmp
[task 2026-03-21T17:34:29.484+00:00] 17:34:29     INFO - REFTEST INFO | Saved app info as D:\task_177411332592375\build\blobber_upload_dir\3dfec1fd-9f5f-4469-acc4-db3217f68f8f.extra
[task 2026-03-21T17:34:29.485+00:00] 17:34:29     INFO - REFTEST PROCESS-CRASH | 3dfec1fd-9f5f-4469-acc4-db3217f68f8f | application crashed [[Unknown]] | gfx/tests/crashtests/1652750-deep-scene-stack.html 
[task 2026-03-21T17:34:29.485+00:00] 17:34:29     INFO - Process type: content
[task 2026-03-21T17:34:29.485+00:00] 17:34:29     INFO - Process pid: None
[task 2026-03-21T17:34:29.486+00:00] 17:34:29     INFO - Crash dump filename: C:\Users\task_177411332592375\AppData\Local\Temp\tmpwy11dunm.mozrunner\minidumps\3dfec1fd-9f5f-4469-acc4-db3217f68f8f.dmp
[task 2026-03-21T17:34:29.952+00:00] 17:34:29     INFO - REFTEST INFO | Result summary:
[task 2026-03-21T17:34:29.952+00:00] 17:34:29     INFO - REFTEST INFO | Successful: 0 (0 pass, 0 load only)
[task 2026-03-21T17:34:29.952+00:00] 17:34:29     INFO - REFTEST INFO | Unexpected: 1 (1 unexpected fail, 0 unexpected pass, 0 unexpected asserts, 0 failed load, 0 exception)
[task 2026-03-21T17:34:29.952+00:00] 17:34:29     INFO - REFTEST INFO | Known problems: 0 (0 known fail, 0 known asserts, 0 random, 0 skipped, 0 slow)
[task 2026-03-21T17:34:29.952+00:00] 17:34:29     INFO - REFTEST SUITE-END | Shutdown
[task 2026-03-21T17:34:30.015+00:00] 17:34:30     INFO - REFTEST INFO | Process mode: e10s
[task 2026-03-21T17:34:30.046+00:00] 17:34:30     INFO - Return code: 245
[task 2026-03-21T17:34:30.060+00:00] 17:34:30  WARNING - Got 1 unexpected crashes
[task 2026-03-21T17:34:30.060+00:00] 17:34:30     INFO - TinderboxPrint: reftest-crashtest<br/>3034/0/0&nbsp;<em class="testfail">CRASH</em>
[task 2026-03-21T17:34:30.060+00:00] 17:34:30  WARNING - setting return code to 2
[task 2026-03-21T17:34:30.060+00:00] 17:34:30     INFO - The reftest suite: crashtest ran with return status: FAILURE
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Running post-action listener: _package_coverage_data
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Running post-action listener: _resource_record_post_action
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Running post-action listener: process_java_coverage_data
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - [mozharness: 2026-03-21 17:34:30.061412Z] Finished run-tests step (success)
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - [mozharness: 2026-03-21 17:34:30.061412Z] Running uninstall step.
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Running pre-action listener: _resource_record_pre_action
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Running main action method: uninstall
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Skipping uninstall for non-MSIX test
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - Running post-action listener: _resource_record_post_action
[task 2026-03-21T17:34:30.061+00:00] 17:34:30     INFO - [mozharness: 2026-03-21 17:34:30.061938Z] Finished uninstall step (success)
[task 2026-03-21T17:34:30.063+00:00] 17:34:30     INFO - Running post-run listener: _resource_record_post_run
[task 2026-03-21T17:34:30.298+00:00] 17:34:30     INFO - Validating Perfherder data against D:\task_177411332592375\mozharness\external_tools\performance-artifact-schema.json
[task 2026-03-21T17:34:30.312+00:00] 17:34:30     INFO - PERFHERDER_DATA: {"framework": {"name": "job_resource_usage"}, "suites": [{"name": "reftest.crashtest.overall", "extraOptions": ["e10s", "taskcluster-Standard_F8s_v2"], "subtests": [{"name": "cpu_percent", "value": 20.607720741357763}, {"name": "io_write_bytes", "value": 2555525120}, {"name": "io.read_bytes", "value": 580293632}, {"name": "io_write_time", "value": 7}, {"name": "io_read_time", "value": 21}]}, {"name": "reftest.crashtest.start-pulseaudio", "subtests": [{"name": "time", "value": 0.016000000000019554}, {"name": "cpu_percent", "value": 0}]}, {"name": "reftest.crashtest.unlock-keyring", "subtests": [{"name": "time", "value": 0.0}, {"name": "cpu_percent", "value": 0}]}, {"name": "reftest.crashtest.install", "subtests": [{"name": "time", "value": 2.9839999999999804}, {"name": "cpu_percent", "value": 16.674479166666664}]}, {"name": "reftest.crashtest.stage-files", "subtests": [{"name": "time", "value": 0.0}, {"name": "cpu_percent", "value": 0}]}, {"name": "reftest.crashtest.run-tests", "subtests": [{"name": "time", "value": 237.65699999999998}, {"name": "cpu_percent", "value": 20.644160353535348}]}, {"name": "reftest.crashtest.uninstall", "subtests": [{"name": "time", "value": 0.0}, {"name": "cpu_percent", "value": 0}]}]}
[task 2026-03-21T17:34:30.314+00:00] 17:34:30     INFO - Total resource usage - Wall time: 241s; CPU: Can't collect data; Read bytes: 580293632; Write bytes: 2555525120; Read time: 21; Write time: 7
[task 2026-03-21T17:34:30.314+00:00] 17:34:30     INFO - TinderboxPrint: I/O read bytes / time<br/>580,293,632 / 21
[task 2026-03-21T17:34:30.314+00:00] 17:34:30     INFO - TinderboxPrint: I/O write bytes / time<br/>2,555,525,120 / 7
[task 2026-03-21T17:34:30.314+00:00] 17:34:30     INFO - TinderboxPrint: CPU idle<br/>1,526.1 (79.1%)
[task 2026-03-21T17:34:30.314+00:00] 17:34:30     INFO - TinderboxPrint: CPU system<br/>158.0 (8.2%)
[task 2026-03-21T17:34:30.314+00:00] 17:34:30     INFO - TinderboxPrint: CPU user<br/>236.9 (12.3%)
[task 2026-03-21T17:34:30.315+00:00] 17:34:30     INFO - start-pulseaudio - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2026-03-21T17:34:30.315+00:00] 17:34:30     INFO - unlock-keyring - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2026-03-21T17:34:30.316+00:00] 17:34:30     INFO - install - Wall time: 3s; CPU: 17%; Read bytes: 12676096; Write bytes: 14229504; Read time: 1; Write time: 0
[task 2026-03-21T17:34:30.316+00:00] 17:34:30     INFO - stage-files - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2026-03-21T17:34:30.329+00:00] 17:34:30     INFO - run-tests - Wall time: 238s; CPU: 21%; Read bytes: 567617536; Write bytes: 2541230080; Read time: 20; Write time: 7
[task 2026-03-21T17:34:30.331+00:00] 17:34:30     INFO - uninstall - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2026-03-21T17:34:31.511+00:00] 17:34:31  WARNING - returning nonzero exit status 2
[taskcluster 2026-03-21T17:34:31.619Z]                        Exit Code: 2
[taskcluster 2026-03-21T17:34:31.619Z]                        User Time: 15.625ms
[taskcluster 2026-03-21T17:34:31.619Z]                      Kernel Time: 31.25ms
[taskcluster 2026-03-21T17:34:31.619Z]                        Wall Time: 6m6.5103046s
[taskcluster 2026-03-21T17:34:31.619Z]  Average Available System Memory: 12.76 GiB
[taskcluster 2026-03-21T17:34:31.619Z]       Average System Memory Used: 3.23 GiB
[taskcluster 2026-03-21T17:34:31.619Z]          Peak System Memory Used: 4.47 GiB
[taskcluster 2026-03-21T17:34:31.619Z]              Total System Memory: 16.00 GiB
[taskcluster 2026-03-21T17:34:31.619Z]                           Result: FAILED
[taskcluster 2026-03-21T17:34:31.619Z] === Task Finished ===
[taskcluster 2026-03-21T17:34:31.620Z] Task Duration: 6m6.5124932s
[taskcluster 2026-03-21T17:34:32.774Z] [mounts] Preserving cache: Moving "D:\\task_177411332592375\\.task-cache\\pip" to "D:\\caches\\KYJHq-k1R42fLFMd6JOQqA"
[taskcluster 2026-03-21T17:34:32.775Z] [mounts] Preserving cache: Moving "D:\\task_177411332592375\\.task-cache\\uv" to "D:\\caches\\BQ24PNE4SjWhL53jFtVWbw"
[taskcluster:error] exit status 2

Comment 1

[BugBot [:suhaib / :marco]]

:Jamie, since you are the author of the regressor, bug 1657973, could you take a look?

For more information, please visit BugBot documentation.

Comment 2

[James Teh [:Jamie]]

:tnikkel, do you think there's any chance this is an instance of bug 1958124? What's really odd here is that for some reason, the log doesn't include a crash stack at all, which is really unhelpful and frustrating. I can't reproduce it locally with a win64 build and I also can't get win32 to build at all for some reason (mach build just hangs forever with no output).

Comment 3

[Timothy Nikkel (:tnikkel)]

(In reply to James Teh [:Jamie] from comment #2)

:tnikkel, do you think there's any chance this is an instance of bug 1958124?

Yeah, that seems likely. I'll try to take a closer look. (leaving needinfo for now)

What's really odd here is that for some reason, the log doesn't include a crash stack at all, which is really unhelpful and frustrating. I can't reproduce it locally with a win64 build and I also can't get win32 to build at all for some reason (mach build just hangs forever with no output).

You could get try server to make you a build and download it via clicking artifacts and debugging tools in the bottom panel, and then finding target.zip.

Comment 4

[James Teh [:Jamie]]

(In reply to Timothy Nikkel (:tnikkel) from comment #3)

You could get try server to make you a build and download it via clicking artifacts and debugging tools in the bottom panel, and then finding target.zip.

Ah, that's true. Thanks. This didn't occur to me because I can't run most mach tests against a build without a corresponding build environment, but this crash test doesn't seem to rely on anything specific to the harness, so I should be able to just load it directly.

Comment 5

[James Teh [:Jamie]]

Curiously, manually loading the test case locally in a downloaded 32 bit build doesn't reproduce the crash, even if I refresh many times. 😕

Comment 6

[Timothy Nikkel (:tnikkel)]

Two potential ways to attack this: in layout/generic/moz.build changed UNIFIED_SOURCES to SOURCES and see if that makes it go away. The regressing patch, did it add local variables to nsIFrame::BuildDisplayList* functions? Those could be taking more stack space.

Comment 7

[James Teh [:Jamie]]

I added 3 local variables to nsIFrame::BuildDisplayListForChild: 2 uint64_t variables and a single pointer. I could split this code into a separate function and call it from BuildDisplayListForChild, but I'm guessing that will just get inlined by the compiler as well.

Comment 8

[Intermittent Failures Robot]

3 failures were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Table

	no_variant
windows11-x86/opt	3

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=2025119&startday=2026-03-16&endday=2026-03-22&tree=all

Comment 9

[James Teh [:Jamie]]

nsIFrame.cpp is already non-unified on Win32. Hmm.

Try run with unchanged main to use as a control
Try run with layout/generic and layout/painting building non-unified

Comment 10

[James Teh [:Jamie]]

(In reply to James Teh [:Jamie] from comment #9)

Try run with layout/generic and layout/painting building non-unified

Surprisingly, switching these to non-unified didn't help: we still see crashes.

Comment 11

[James Teh [:Jamie]]

Try run with nsIFrame/gfx changes reverted
Try run with nsIFrame accId variables removed

Comment 12

[James Teh [:Jamie]]

Okay. So the issue is definitely triggered by the addition of the variables I noted in comment 7. Given that switching to non-unified doesn't seem to fix this (and nsIFrame.cpp was already non-unified anyway), I'm looking into other approaches, starting with moving this code into a static function, though I'm slightly concerned the compiler will just inline it.

Try run with nsIFrame variables moved into a separate function
Try run with this + MOZ_NEVER_INLINE

Comment 13

[Timothy Nikkel (:tnikkel)]

If that approach is too difficult or not working another option is to finally just decide that we will crash on these kinds of extreme files in the declining population of win32. We'd just create a slightly less deep scene crash test that we run on win32 to prevent it from regressing any further.

Comment 14

[James Teh [:Jamie]]

Try run: never inline MaybeAddAccId but only on Win32

Comment 15

[James Teh [:Jamie]]

Attachment: Bug 2025119: nsIFrame::BuildDisplayListForChild: On Win32, don't inline the code which adds nsDisplayAccessibleId. r?tnikkel

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/12c9cb496b58
https://hg.mozilla.org/integration/autoland/rev/0a83200ae101
nsIFrame::BuildDisplayListForChild: On Win32, don't inline the code which adds nsDisplayAccessibleId. r=tnikkel,layout-reviewers

Comment 17

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/0a83200ae101

Comment 18

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1657973

Comment 19

[BugBot [:suhaib / :marco]]

The patch landed in nightly and beta is affected.
:Jamie, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
  • See https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift for documentation on how to request an uplift.
• If no, please set status-firefox150 to wontfix.

For more information, please visit BugBot documentation.

Comment 20

[Phabricator Automation]

firefox-beta Uplift Approval Request

• User impact if declined/Reason for urgency: Crash (albeit very unlikely to be hit in the wild) on 32 bit Windows builds.
• Code covered by automated testing?: yes
• Fix verified in Nightly?: yes
• Needs manual QE testing?: no
• Steps to reproduce for manual QE testing:
• Risk associated with taking this patch: low
• Explanation of risk level: Just separates out some code into a function.
• String changes made/needed?: none
• Is Android affected?: no

Comment 21

[James Teh [:Jamie]]

Attachment: Bug 2025119: nsIFrame::BuildDisplayListForChild: On Win32, don't inline the code which adds nsDisplayAccessibleId.

Original Revision: https://phabricator.services.mozilla.com/D289442

Comment 22

[Pulsebot]

https://github.com/mozilla-firefox/firefox/commit/ccc711d72b79
https://hg.mozilla.org/releases/mozilla-beta/rev/4c98e2d496a1