1750616 - :screenshot in devtools huge memory leak

Status: RESOLVED WONTFIX

(DevTools :: General, defect)

Description

[remimarche]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0

Steps to reproduce:

To reproduce

1. Open any webpage and the devtools.
2. Type screenshot --dpr 10 (the --dpr 4 is to be sure you get a full resolution heavy image)
   1. If the resulting screenshot is big enough, it should take a few seconds for the screenshot to be taken.
   2. If it is not the case, zoom in to make the resulting images even bigger.
3. Press up arrow and Enter a few times (30 is a good starting number), to rerun the command multiple times.
4. The first image should be taken only when you are done with submitting new commands to the console.
   1. If it is not the case it means you were not fast enough to press UpArrow Enter
   2. If it still doesn't work, try the AutoHotKey script at the end.
5. Open Task manager or the operating system's equivalent
6. Notice how huge the RAM consumption is.

On my machine, it went up to 9GB, with 3 tabs open (single window).

AutoHotKey Script

https://www.autohotkey.com/

To run it: copy paste the following code in a new file with extension .ahk
Run it with autohotkey (if ahk is installed, then this should correspond to double clicking the newly created file in your file manager)
Go back to the console devtools and press Ctrl+Alt+J
To cancel the running script, press and hold escape until it stops.

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
#Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

^!j::
count := 0
While (count < 63 And Not GetKeyState("Escape")) {
    ; https://developer.mozilla.org/en-US/docs/Tools/Taking_screenshots
    ScreenshotString := Format(":screenshot --filename screenshot{1} --dpr 10", count)
    Send, %ScreenshotString%
    Send, {Enter}
    count := count + 1
}
return

Actual results:

A huge persisting RAM usage

Expected results:

No more usage after the screenshots than before

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Firefox::Screenshots' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Julian Descottes [:jdescottes]]

Nicolas: Alex suggested to write a memory test (start in the allocations folder!).

Comment 3

[remimarche]

Attachment: video.zip

Here's a screencast of me taking 62 captures in a row with the help of a script.

All the commands are issued to the console, then there's some pause and nothing seems to happen for a while, while the browser is preparing itself for getting the screenshots, then, a sort of flash occurs, along with a weird sound (the sound of 62 screenshots being taken at the same time). After that, the RAM usage rapidly increases (in this case up to 6GB) and never comes down. At first I thought it was a matter of time, and it would end up going down, but it doesn't. It's been 15 minutes, and the memory usage is still at 6.18 GB.

Comment 4

[remimarche]

I had to zip the video because it was more than 10MB.

I don't know if it has to do with the fact that I'm zoomed when taking the pictures.

Comment 5

[Julian Descottes [:jdescottes]]

Nicolas will try to repro on windows and check if the logger is using GC

Comment 6

[Timothy Nikkel (:tnikkel)]

Can you goto about:memory when your memory is high and save a memory report?

And then click the minimize memory button and see if the memory use goes down.

Comment 7

[Timothy Nikkel (:tnikkel)]

When I do a bunch of screenshots my heap-unclassified in the parent process grows about 20mb for every screenshot I take.

DMD reports the vast majority of this is:

Unreported {
19 blocks in heap block record 1 of 12,310
318,767,104 bytes (318,767,104 requested / 0 slop)
Individual block sizes: 16,777,216 x 19
50.94% of the heap (50.94% cumulative)
67.77% of unreported (67.77% cumulative)
Allocated at {
#01: replace_malloc(unsigned long) (/Applications/Firefox Nightly.app/Contents/MacOS/libmozglue.dylib + 0x590a9)
#02: Gecko_StartBulkWriteString (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x4ea0e8)
#03: nsstring_fallible_append_utf8_impl (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x242fa14)
#04: nsXPCComponents_Utils::ReportError(JS::Handle<JS::Value>, JS::Handle<JS::Value>, JSContext*) (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x88572c)
#05: NS_InvokeByIndex (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x1c8e)
#06: XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x8a9041)
#07: XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x8aac3d)
#08: Interpret(JSContext*, js::RunState&) (/Applications/Firefox Nightly.app/Contents/MacOS/XUL + 0x169b7ee)
}
}

Putting a "xpc_DumpJSStack(true, true, true);" into nsXPCComponents_Utils::ReportError I get

0 setMetadata() ["resource://gre/modules/DownloadHistory.jsm":327:9]
1 InterpretGeneratorResume(gen = "[object Object]", val = "undefined", kind = ""next"") ["self-hosted":1475:33]
2 AsyncFunctionNext(val = "undefined") ["self-hosted":685:26]
this = [object Object]

So the reportError call is here

https://searchfox.org/mozilla-central/rev/66b43502bdbe4b62cbc8b0c8ff10efc5707352e4/toolkit/components/downloads/DownloadHistory.jsm#327

the url is the data uri of the screenshot, which explains the memory usage.

Questions I don't know the answer to:

Why does the PlacesUtils.history.update call fail? Should we avoid the fail somehow?

Why aren't we cleaning up the memory from nsXPCComponents_Utils::ReportError?

Comment 8

[remimarche]

Same here: Heap_unclassified grows to a few Gb.

Some more info: It grows more rapidly if there are many images on the page. This bug report page, for example, needs a --dpr=1000 for the screenshot to be taken even with a slight delay. In comparison, go to this one, then zoom in to 400% on the image on the left and take the screenshots.

Comment 9

[Timothy Nikkel (:tnikkel)]

The err message is
PageInfo: Invalid value for property 'url': "data:image/png;base64,iVBORw0...

so that much be coming from
https://searchfox.org/mozilla-central/rev/66b43502bdbe4b62cbc8b0c8ff10efc5707352e4/toolkit/components/places/PlacesUtils.jsm#766,802

Comment 10

[Timothy Nikkel (:tnikkel)]

nsXPCComponents_Utils::ReportError

https://searchfox.org/mozilla-central/rev/fe3fa7f53037d4e869858fef4ff9310dfa795c41/js/xpconnect/src/XPCComponents.cpp#1588

calls nsConsoleService::LogMessageWithMode, there are no listeners in the parent process, so we just buffer the message until we hit I think 250 buffered messages

https://searchfox.org/mozilla-central/rev/66b43502bdbe4b62cbc8b0c8ff10efc5707352e4/xpcom/base/nsConsoleService.cpp#360

So we should maybe truncate long urls when we report errors. And maybe have a limit on the size of buffered error messages in addition to a limit on the number of error messages.

Comment 11

[Timothy Nikkel (:tnikkel)]

Oh, and we should also instrument this for memory reporting maybe.

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 13

[Nicolas Chevobbe [:nchevobbe]]

Thanks Timothy for the investigation.
I'm not seeing the error message anymore, but this is almost 3 years old now, so things might have changed.
I can't confirm the bug is still here or not though, using about:memory Measure button I'm able to see memory increasing even with a simple firefox window having only an about:blank tab :/

remimarche , are you still seeing this bug?

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:Honza, since the bug has recent activity, could you have a look please?

For more information, please visit BugBot documentation.

Comment 15

[Julian Descottes [:jdescottes]]

Can't reproduce the issue