1939295 - Lots of ghost windows accumulating for google origins via FetchStreamReader

Status: VERIFIED FIXED

(Core :: DOM: Networking, defect, P2)

Description

[Katelyn Gadd (:kael)]

Attachment: anonymized memory report

There seems to be a regression either in FF or on Google's end that specifically causes the webIsolateds for google.com origins to accumulate lots of ghost windows. It seems like almost every Google-origin tab I open and then close becomes a ghost - maps, search, gdocs, drive folders, spreadsheets. Nothing seems to make them go away, some of them are hours or days old. I tried GC, CC and minimize memory usage.

I don't see ghost windows for any other origins, just google. Each one is using 30-100mb of memory. Due to these I have 3 google webIsolates, using a total of around 2.7GB of RAM, for a very small number of open google tabs.

YouTube isolates don't seem to have this problem, other than a couple very cheap (<10mb) ghost embeds I see. So if YouTube has the same problem it's not nearly as bad.

I can provide a de-anonymized memory report or gc logs or whatever if you let me know what you need.

Comment 1

[Katelyn Gadd (:kael)]

Attachment: about:support

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::JavaScript: GC' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Andrew McCreight [:mccr8]]

Thanks for the report. This sounds bad. I don't see it in my local session, for what it is worth.

Can you reproduce this with WebExtensions disabled? Or Troubleshoot mode?

Failing that, the next step would be to look at CC logs. You can generate these from "Save concise" in about:memory. The file names have the PID in them. You'd probably want the smallest logs from a process experiencing the problem. The logs contain a lot of information about what is in the process (eg the GC logs contain the contents of all JS strings) so I'm not sure what the best way is to get the logs to me or whatever. If you want, you could run my find_roots.py script yourself on the log, from here. The first argument to run it is the CC log file name and for the second one you'd want to enter nsGlobalWindowInner. That will hopefully give some information about why all of the windows are alive. We might need "verbose" logs, depending on how it turns out.

Comment 4

[Katelyn Gadd (:kael)]

Thanks for taking a look. I've shared a dropbox folder with you that contains concise and verbose gc/cc logs, along with a full non-anonymized memory report. The processes for google.com are the problem ones; they should have PIDs in the memory report I think.

I tested disabling all the extensions that looked related and then doing GC/CC, but that's obviously not as good as running without extensions. I'm not sure how realistic it is for my workflow to spend multiple days without adblock or tree style tab but I can see if I can manage it.

I just did a new memory measurement and it looks like I've accumulated more new ghost windows since reporting this, so it's possible I could reproduce this in a new profile. I'll try to make time to do that test in the next week or so.

What is Troubleshoot mode? I'm unfamiliar with it.

Comment 5

[Andrew McCreight [:mccr8]]

(In reply to Katelyn Gadd (:kael) from comment #4)

What is Troubleshoot mode? I'm unfamiliar with it.

Formerly known as Safe Mode. Basically it disables extensions, hardware acceleration and maybe some other things.

Comment 6

[Andrew McCreight [:mccr8]]

I looked at the CC logs that kael shared. All of the leaked windows, including the ones for YouTube, were being leaked via various FetchStreamReader, either directly (eg the inner window was the mGlobal of a FetchStreamReader) or indirectly (eg the window was being held alive via an event in the ELM of a window mGlobal of a FetchStreamReader).

The FetchStreamReader all had a refcount of 2. They all had one known reference, which was the mFetchStreamReader field of a ReadRequest.

Hopefully somebody who is more familiar with FetchStreamReader can figure out what the missing reference might be.

Comment 7

[Katelyn Gadd (:kael)]

I verified that some searches I made just now became ghost windows, and then noticed that uBlock is enabled for google search pages. I've tried disabling it for them and will report back if it makes the issue go away.

I didn't see any sign of it using FetchStreamReader in its source though (I grepped for .body)

Comment 8

[Kagami Rosylight [:saschanaz] (they/them)]

(In reply to Andrew McCreight [:mccr8] from comment #6)

I looked at the CC logs that kael shared. All of the leaked windows, including the ones for YouTube, were being leaked via various FetchStreamReader, either directly (eg the inner window was the mGlobal of a FetchStreamReader) or indirectly (eg the window was being held alive via an event in the ELM of a window mGlobal of a FetchStreamReader).

The FetchStreamReader all had a refcount of 2. They all had one known reference, which was the mFetchStreamReader field of a ReadRequest.

Hopefully somebody who is more familiar with FetchStreamReader can figure out what the missing reference might be.

FetchStreamReader is passed to nsIAsyncOutputStream::AsyncWait, perhaps that's the culprit and somehow the stream is not being closed nor releasing the reference?

Comment 9

[Kagami Rosylight [:saschanaz] (they/them)]

(Comment #0 says YT is not affected heavily but still a bit. That might not be true for others. I still need to investigate more though.)

Comment 10

[Andrew McCreight [:mccr8]]

Rob, do you have any ideas how uBlock Origin enabled for some specific set of pages might end up using FetchStreamReader? Maybe that happens at the WebExtension level somehow? Thanks.

Comment 11

[Kagami Rosylight [:saschanaz] (they/them)]

I can't repro with or without ublock 🤔, everything goes away from about:memory measurement when I close the tabs so far.

Comment 12

[Kagami Rosylight [:saschanaz] (they/them)]

Katelyn, do you have other extensions than uBlock, so that we can try with them too?

Comment 13

[Andrew McCreight [:mccr8]]

(In reply to Kagami Rosylight [:saschanaz] (they/them) from comment #11)

I can't repro with or without ublock 🤔, everything goes away from about:memory measurement when I close the tabs so far.

You need to prevent the content process that the page is from being killed off when you close the tab. First, open Google/YouTube in 4 separate tabs, then open a new tab to do your actual testing in. To be extra sure, you can confirm that the process your test page is in does not go away after you close it, by writing down the PID.

Comment 14

[Katelyn Gadd (:kael)]

https://bugzilla.mozilla.org/show_bug.cgi?id=1939295#c1 lists the extensions I had installed at the time I made the report. At the time, uBlock was enabled for google.com but not for some of the subdomains (mail had it disabled, docs had it disabled, calendar had it enabled, etc). I've since disabled it for google.com and haven't gotten a new ghost window yet, but it's too soon to say whether that was the fix.

Out of the extensions I have installed tree style tab is probably the most likely one to be involved, none of them should really be engaged while performing google searches other than uBlock (which is probably blocking search ads?). To summarize the list here:

1. 10ten Japanese Reader
   AFAIK this isn't active in a page until you explicitly summon it.
2. Firefox Color
3. Owl
   Owl is not enabled for google search tabs, google mail tabs, or google calendar tabs, so I don't think it's in play here.
4. Tree Style Tab
5. uBlock Origin
   Probably was blocking search ads, but not all the ghost windows were searches, just many of them. I've gotten ghost windows for drive, sheets and maps.

The rest of the extensions in the about:support list are disabled.

Comment 15

[Rob Wu [:robwu]]

It is likely independent of extensions.

Tree Style Tabs does not use fetch, let alone FetchStreamReader in web content.

uBlock Origin has some references to fetch in its source (including wrapping the fetch global in some cases, all in the context of web content), but no use of FetchStreamReader either.
If you think that uBlock Origin activated something meaningfully, open its Logger to see which blocking rules applied.

Comment 16

[Kagami Rosylight [:saschanaz] (they/them)]

Somehow I never see lots of ghosts windows even with dom.ipc.processCount.webIsolated=1, but I managed to get one at least with https://docs.google.com/ main screen, signing in with Mozilla LDAP account, without any extension on a fresh profile.

If you attached the debugger, you can see the FetchStreamReader::StartConsuming is called frequently but FetchStreamReader::CloseAndRelease is not. So I made a repro with the same situation:

<!DOCTYPE html>
<meta charset="utf-8" />
<script type="module">
  console.log(await new Response(new ReadableStream({
    pull() {
      const { promise, resolve } = Promise.withResolvers();
      window.r = resolve;
      return promise;
    }
  })).text());
</script>

And closing that tab makes it a ghost 😓

This feels like a reverse of bug 1849860; while fixing that bug we had to make InputToReadableStreamAlgorithms to inherit from GlobalTeardownObserver, as we had to cut the cycle manually as closing the tab do not automatically disconnect the network streams. We do have similar mechanism in FetchStreamReader via WorkerRef but we don't have the equivalent for non-worker contexts.

Comment 17

[Katelyn Gadd (:kael)]

I reproduced with uBlock disabled, it just isn't 100% on every opened tab. I should have specified that i'm logged into my google account, which has a custom domain (so it's a google workspaces account, not a regular gmail account). It's possible that all my google tabs are communicating with the network (or a service worker?) and if I close them at the right time that's causing this. It would explain a lot of them being google search results pages since I often find what I'm looking for pretty quickly and navigate away.

Comment 18

[Kagami Rosylight [:saschanaz] (they/them)]

This can also be a general memory leak too:

let stream = new ReadableStream();
new Response(stream).text();
stream = new WeakRef(stream);
// stream object never goes away;

It's because FetchStreamReader::Create() creates a pipe through NS_NewPipe2 but that pipe will not be closed by anyone and is not part of cycle collection either. InputStreamHolder has been introduced to solve the exact issue, the same approach should work here too.

Interestingly Chrome seems to be affected by the same issue.

Comment 19

[Kagami Rosylight [:saschanaz] (they/them)]

Attachment: Bug 1939295 - Add OutputStreamHolder r=jesup

Comment 20

[Kagami Rosylight [:saschanaz] (they/them)]

Ideally bug 1825763 would be able to reduce a lot of code, but for now I'm doing minimal fix to make uplift easier.

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4ee7d8aa407d
Add OutputStreamHolder r=jesup

Comment 22

[Kagami Rosylight [:saschanaz] (they/them)]

We are now landing the patch and will see if it helps other performance regression reports for YouTube. (See bug 1931717 for the list of the reports.) And I want to use this place to...

... send my thank you to the reporter Katelyn! Ever since we had the initial report of OOM this is the first time we saw the concrete proof of Firefox having internal memory leak on Google/YouTube. We had hard time identifying the cause and this report allowed us to take a swift action!

So thank you Katelyn for using your precious time for us to proceed!

Comment 23

[Frederik Braun [:freddy]]

Are we looking at uplifting this or are we still hypothesizing if this will fix it?

Comment 24

[Kagami Rosylight [:saschanaz] (they/them)]

We want to check the ghost-window telemetry and see if the new nightly actually makes it lower.

Comment 25

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/4ee7d8aa407d

Comment 26

[Kagami Rosylight [:saschanaz] (they/them)]

Attachment: Bug 1939295 - Add OutputStreamHolder r=jesup

Similar to InputStreamHolder, this adds OutputStreamHolder to FetchStreamReader:

1. OutputStreamHolder is not part of the cycle collection but is freed when FetchStreamReader goes away
2. nsIAsyncOutputStream holds OutputStreamHolder which only weakly hold FetchStreamReader, allowing FetchStreamReader to be cycle collected.
3. GlobalTeardownObserver is not added here as we only accept JS ReadableStream here instead of nsIInputStream, which is part of the cycle collection unlike nsIInputStream.

Original Revision: https://phabricator.services.mozilla.com/D233553

Comment 27

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: Users would still get bad performance caused by memory leak on google/youtube
• Code covered by automated testing: yes
• Fix verified in Nightly: yes
• Needs manual QE test: no
• Steps to reproduce for manual QE testing: (One can still try: 1) set dom.ipc.processCount.webIsolated=1 2) open https://docs.google.com and sign in 3) duplicate the tab 4) switch to the first one and then close it 5) go to about:memory and check there's any ghost window
• Risk associated with taking this patch: Low
• Explanation of risk level: Good test coverage
• String changes made/needed: None
• Is Android affected?: yes

Comment 28

[Kagami Rosylight [:saschanaz] (they/them)]

Attachment: Bug 1939295 - Add OutputStreamHolder r=jesup

Similar to InputStreamHolder, this adds OutputStreamHolder to FetchStreamReader:

1. OutputStreamHolder is not part of the cycle collection but is freed when FetchStreamReader goes away
2. nsIAsyncOutputStream holds OutputStreamHolder which only weakly hold FetchStreamReader, allowing FetchStreamReader to be cycle collected.
3. GlobalTeardownObserver is not added here as we only accept JS ReadableStream here instead of nsIInputStream, which is part of the cycle collection unlike nsIInputStream.

Original Revision: https://phabricator.services.mozilla.com/D233553

Comment 29

[Kagami Rosylight [:saschanaz] (they/them)]

Attachment: Bug 1939295 - Add OutputStreamHolder r=jesup

Similar to InputStreamHolder, this adds OutputStreamHolder to FetchStreamReader:

1. OutputStreamHolder is not part of the cycle collection but is freed when FetchStreamReader goes away
2. nsIAsyncOutputStream holds OutputStreamHolder which only weakly hold FetchStreamReader, allowing FetchStreamReader to be cycle collected.
3. GlobalTeardownObserver is not added here as we only accept JS ReadableStream here instead of nsIInputStream, which is part of the cycle collection unlike nsIInputStream.

Original Revision: https://phabricator.services.mozilla.com/D233553

Comment 30

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/aeadf10f2355

Comment 31

[Kagami Rosylight [:saschanaz] (they/them)]

Oops, accidentally overwrote on esr128 patch!

Comment 32

[Kagami Rosylight [:saschanaz] (they/them)]

Attachment: Bug 1939295 - Add OutputStreamHolder r=jesup

Similar to InputStreamHolder, this adds OutputStreamHolder to FetchStreamReader:

1. OutputStreamHolder is not part of the cycle collection but is freed when FetchStreamReader goes away
2. nsIAsyncOutputStream holds OutputStreamHolder which only weakly hold FetchStreamReader, allowing FetchStreamReader to be cycle collected.
3. GlobalTeardownObserver is not added here as we only accept JS ReadableStream here instead of nsIInputStream, which is part of the cycle collection unlike nsIInputStream.

Original Revision: https://phabricator.services.mozilla.com/D233553

Comment 33

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-release/rev/ea241be5664d

Comment 35

[Bogdan Maris, Desktop Test Engineering]

I tried to reproduce the initial issue (seeing the ghost windows) using an affected build (Nightly from 2024-12-27 and Firefox 134.0b10) on Windows 11 and MacOS 13 but I am not sure how to see a 'ghost window' in about:memory or anywhere else. I used the steps from comment 27 with the value 1 or 2 of the dom.ipc.processCount.webIsolated pref but after I go to about:memory and measure, I have the following processes open for me:

Process index
• Main Process (pid 8836) 
• weblsolated=https://google.com (pid 7328)
• GPU (pid 2864)
• privilegedabout (pid 1968) 
• extension (pid 1932)
• prealloc (pid 11796)
• prealloc (pid 8664)
• prealloc (pid 7204) RDD (pid 764)
• Utility (pid 9356, sandboxingKind 1) 
• Utility (pid 4904, sandboxingKind 0) 
• Socket (pid 9440)

Not sure if I should have another • weblsolated=https://google.com process open or something else. A bit more info would be helpful.

Comment 36

[Kagami Rosylight [:saschanaz] (they/them)]

(In reply to Bogdan Maris, Desktop Test Engineering [PTO until 13 Jan] from comment #35)

I tried to reproduce the initial issue (seeing the ghost windows) using an affected build (Nightly from 2024-12-27 and Firefox 134.0b10) on Windows 11 and MacOS 13 but I am not sure how to see a 'ghost window' in about:memory or anywhere else. I used the steps from comment 27 with the value 1 or 2 of the dom.ipc.processCount.webIsolated pref but after I go to about:memory and measure, I have the following processes open for me:

It would be under • weblsolated=https://google.com (pid 7328). Note that the definition of a ghost window is a window being detached for a minute and still not being purged, so an extra step is to wait for a minute after closing tabs.

Before the fix you would see either:

webIsolated=https://google.com (pid 16952)
Explicit Allocations

245.83 MB (100.0%) -- explicit
├──103.77 MB (42.21%) -- window-objects
│  ├───82.61 MB (33.61%) ++ top(https://docs.google.com/document/u/0/?tgif=d, id=53)
│  └───21.16 MB (08.61%) -- top(none)/detached
│      ├──16.71 MB (06.80%) -- window(https://docs.google.com/document/u/0/?tgif=d)
│      │  ├──14.69 MB (05.98%) -- js-realm(https://docs.google.com/document/u/0/?usp=direct_url)
│      │  │  ├──10.21 MB (04.15%) -- classes
│      │  │  │  ├───3.50 MB (01.42%) -- class(Function)/objects
│      │  │  │  │   ├──3.14 MB (01.28%) ── gc-heap
│      │  │  │  │   └──0.36 MB (00.15%) ── malloc-heap/slots
│      │  │  │  ├───2.98 MB (01.21%) ++ class(Object)/objects
│      │  │  │  ├───2.74 MB (01.11%) ++ class(Array)/objects
│      │  │  │  └───0.98 MB (00.40%) ++ (10 tiny)
│      │  │  ├───3.71 MB (01.51%) ++ scripts
│      │  │  └───0.78 MB (00.32%) ++ (6 tiny)
│      │  └───2.02 MB (00.82%) ++ (3 tiny)
│      ├───2.49 MB (01.01%) ++ window(https://ogs.google.com/u/0/widget/app?awwd=1&origin=https%3A%2F%2Fdocs.google.com&cn=app&pid=25&spid=25&hl=en)
│      └───1.96 MB (00.80%) ++ (5 tiny)

And one minute later it becomes a ghost:

116.39 MB (100.0%) -- explicit
├───51.05 MB (43.87%) -- window-objects
│   ├──34.39 MB (29.54%) ++ top(https://docs.google.com/document/u/0/?tgif=d, id=77)
│   └──16.67 MB (14.32%) -- top(none)
│      ├──16.53 MB (14.20%) -- ghost
│      │  ├──15.36 MB (13.20%) -- window(https://docs.google.com/document/u/0/?tgif=d)
│      │  │  ├──13.34 MB (11.46%) -- js-realm(https://docs.google.com/document/u/0/?tgif=d)
│      │  │  │  ├───9.61 MB (08.25%) -- classes
│      │  │  │  │   ├──3.47 MB (02.98%) -- class(Function)/objects
│      │  │  │  │   │  ├──3.11 MB (02.68%) ── gc-heap
│      │  │  │  │   │  └──0.36 MB (00.31%) ── malloc-heap/slots
│      │  │  │  │   ├──2.82 MB (02.42%) -- class(Object)/objects
│      │  │  │  │   │  ├──1.51 MB (01.30%) ── gc-heap
│      │  │  │  │   │  └──1.31 MB (01.12%) ++ malloc-heap
│      │  │  │  │   ├──2.44 MB (02.10%) -- class(Array)/objects
│      │  │  │  │   │  ├──1.58 MB (01.36%) -- malloc-heap
│      │  │  │  │   │  │  ├──1.40 MB (01.20%) ── elements/normal
│      │  │  │  │   │  │  └──0.18 MB (00.16%) ── slots
│      │  │  │  │   │  └──0.86 MB (00.74%) ── gc-heap
│      │  │  │  │   └──0.88 MB (00.76%) ++ (8 tiny)
│      │  │  │  ├───3.72 MB (03.20%) -- scripts
│      │  │  │  │   ├──1.96 MB (01.68%) ── gc-heap
│      │  │  │  │   └──1.76 MB (01.51%) ── malloc-heap/data
│      │  │  │  └───0.01 MB (00.01%) ++ (2 tiny)
│      │  │  └───2.02 MB (01.74%) ++ (3 tiny)
│      │  └───1.16 MB (01.00%) ++ (2 tiny)
│      └───0.14 MB (00.12%) -- detached/window(about:blank)
│          ├──0.14 MB (00.12%) ++ js-realm(https://docs.google.com/document/u/0/?tgif=d, about:blank)
│          ├──0.00 MB (00.00%) ++ dom
│          └──0.00 MB (00.00%) ── layout/style-sheets

Comment 37

[Bogdan Maris, Desktop Test Engineering]

Thanks for the extra info! I managed to reproduce it using both Nightly from 2024-12-27 and Firefox 134.0b10. Logs bellow:

webIsolated=https://google.com (pid 7828)
Explicit Allocations

115.50 MB (100.0%) -- explicit
├───49.39 MB (42.76%) -- window-objects
│   ├──34.60 MB (29.96%) -- top(https://docs.google.com/document/u/0/, id=42)
│   │  ├──25.93 MB (22.45%) -- active
│   │  │  ├──21.47 MB (18.59%) -- window(https://docs.google.com/document/u/0/)
│   │  │  │  ├──11.89 MB (10.29%) -- js-realm(https://docs.google.com/document/u/0/)
│   │  │  │  │  ├───8.01 MB (06.93%) -- classes
│   │  │  │  │  │   ├──3.46 MB (03.00%) -- class(Function)/objects
│   │  │  │  │  │   │  ├──3.11 MB (02.70%) ── gc-heap
│   │  │  │  │  │   │  └──0.35 MB (00.30%) ── malloc-heap/slots
│   │  │  │  │  │   ├──2.54 MB (02.20%) -- class(Object)/objects
│   │  │  │  │  │   │  ├──1.29 MB (01.12%) ── gc-heap
│   │  │  │  │  │   │  └──1.25 MB (01.08%) ++ malloc-heap
│   │  │  │  │  │   └──2.01 MB (01.74%) ++ (11 tiny)
│   │  │  │  │  ├───3.55 MB (03.07%) -- scripts
│   │  │  │  │  │   ├──1.96 MB (01.70%) ── gc-heap
│   │  │  │  │  │   └──1.58 MB (01.37%) ── malloc-heap/data
│   │  │  │  │  └───0.33 MB (00.29%) ++ (5 tiny)
│   │  │  │  ├───8.48 MB (07.34%) -- layout
│   │  │  │  │   ├──3.97 MB (03.44%) ── style-sheets
│   │  │  │  │   ├──1.55 MB (01.34%) ++ (9 tiny)
│   │  │  │  │   ├──1.49 MB (01.29%) ++ style-structs
│   │  │  │  │   └──1.48 MB (01.28%) -- style-sets
│   │  │  │  │      ├──1.47 MB (01.27%) ++ stylist
│   │  │  │  │      └──0.01 MB (00.01%) ── other
│   │  │  │  └───1.11 MB (00.96%) ++ (2 tiny)
│   │  │  ├───3.20 MB (02.77%) -- window(https://ogs.google.com/u/0/widget/app?awwd=1&origin=https%3A%2F%2Fdocs.google.com&cn=app&pid=25&spid=25&hl=en)
│   │  │  │   ├──2.08 MB (01.80%) -- js-realm(https://ogs.google.com/u/0/widget/app?awwd=1&origin=https%3A%2F%2Fdocs.google.com&cn=app&pid=25&spid=25&hl=en)
│   │  │  │   │  ├──1.43 MB (01.24%) ++ classes
│   │  │  │   │  └──0.65 MB (00.56%) ++ (4 tiny)
│   │  │  │   └──1.12 MB (00.97%) ++ (3 tiny)
│   │  │  └───1.26 MB (01.09%) ++ (3 tiny)
│   │  └───8.67 MB (07.51%) -- js-zone(0x1223d3a00)
│   │      ├──2.59 MB (02.25%) ++ (14 tiny)
│   │      ├──2.26 MB (01.95%) ++ property-maps
│   │      ├──2.21 MB (01.91%) -- scopes
│   │      │  ├──1.35 MB (01.17%) ── malloc-heap
│   │      │  └──0.86 MB (00.75%) ── gc-heap
│   │      └──1.61 MB (01.40%) ── unused-gc-things
│   └──14.78 MB (12.80%) -- top(none)
│      ├──14.66 MB (12.69%) -- ghost
│      │  ├──13.49 MB (11.68%) -- window(https://docs.google.com/document/u/0/)
│      │  │  ├──11.62 MB (10.06%) -- js-realm(https://docs.google.com/document/u/0/?usp=direct_url)
│      │  │  │  ├───8.06 MB (06.98%) -- classes
│      │  │  │  │   ├──3.46 MB (03.00%) -- class(Function)/objects
│      │  │  │  │   │  ├──3.12 MB (02.70%) ── gc-heap
│      │  │  │  │   │  └──0.34 MB (00.30%) ── malloc-heap/slots
│      │  │  │  │   ├──2.55 MB (02.21%) -- class(Object)/objects
│      │  │  │  │   │  ├──1.29 MB (01.12%) ── gc-heap
│      │  │  │  │   │  └──1.26 MB (01.09%) ++ malloc-heap
│      │  │  │  │   └──2.05 MB (01.77%) ++ (9 tiny)
│      │  │  │  ├───3.54 MB (03.07%) -- scripts
│      │  │  │  │   ├──1.96 MB (01.70%) ── gc-heap
│      │  │  │  │   └──1.58 MB (01.37%) ── malloc-heap/data
│      │  │  │  └───0.02 MB (00.01%) ++ (2 tiny)
│      │  │  └───1.87 MB (01.62%) ++ (3 tiny)
│      │  └───1.17 MB (01.01%) ++ (2 tiny)
│      └───0.13 MB (00.11%) ++ detached/window(about:blank)

...

4,462 (100.0%) -- event-counts
└──4,462 (100.0%) -- window-objects
   ├──2,577 (57.75%) -- top(https://docs.google.com/document/u/0/, id=42)/active
   │  ├──2,210 (49.53%) -- window(https://docs.google.com/document/u/0/)/dom
   │  │  ├──1,973 (44.22%) ── event-listeners
   │  │  └────237 (05.31%) ── event-targets
   │  ├────361 (08.09%) -- window(https://ogs.google.com/u/0/widget/app?awwd=1&origin=https%3A%2F%2Fdocs.google.com&cn=app&pid=25&spid=25&hl=en)/dom
   │  │    ├──275 (06.16%) ── event-listeners
   │  │    └───86 (01.93%) ── event-targets
   │  └──────6 (00.13%) ++ (3 tiny)
   └──1,885 (42.25%) -- top(none)/ghost/window(https://docs.google.com/document/u/0/)/dom
      ├──1,884 (42.22%) ── event-listeners
      └──────1 (00.02%) ── event-targets

...

3 (100.0%) -- ghost-windows
├──1 (33.33%) ── about:blank
├──1 (33.33%) ── https://docs.google.com/_/og/bscframe
└──1 (33.33%) ── https://docs.google.com/document/u/0/
...
     0.00 MB ── gfx-textures
     0.00 MB ── gfx-textures-peak
     0.00 MB ── gfx-tiles-waste
           3 ── ghost-windows
    65.30 MB ── heap-allocated
     1.00 MB ── heap-chunksize
           0 ── imagelib-surface-cache-already-present-count
    16.91 MB ── imagelib-surface-cache-estimated-locked
    16.91 MB ── imagelib-surface-cache-estimated-total
...

End of webIsolated=https://google.com (pid 7828)

Verified that using the latest RC build 134.0.1 no ghost-windows are recorded across platforms (Windows 10, MacOS 13 and Ubuntu 22.04). Will also verify on Beta and Nightly tomorrow first time.

Comment 38

[Andrew McCreight [:mccr8]]

For future reference, clicking on "minimize memory usage" in about:memory should, I believe, accelerate the ghost window timer and avoid the need to wait for a minute. But it is even better if you are able to verify the fix without doing that, as clicking the button does not match the normal behavior users will get in some situations.

Comment 40

[Bogdan Maris, Desktop Test Engineering]

Also verified that this is fixed on the latest builds of Nightly 136.0a1 and Beta 135.0b4 across platforms (Windows 11, macOS 13 and Ubuntu 22.04). Removing the qa-verify+ flag but will revisit once the fix lands in ESR builds.

Comment 43

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr128/rev/731c59b12de6

Comment 44

[Ryan VanderMeulen [:RyanVM]]

Per https://phabricator.services.mozilla.com/D233820#8135732, setting ESR115 to wontfix.

Comment 45

[Bogdan Maris, Desktop Test Engineering]

Looks good to me using Firefox 128.7.0esr across platforms (macOS 13, Windows 11 and Ubuntu 22.04) as well.