1625262 - (from 800MB heap-unclassified) unreported 86MB from Http2Session::WriteSegmentsAgain

Status: NEW

(Core :: Networking: HTTP, defect, P3)

Description

[Steve Fink [:sfink] [:s:]]

While working on a microbenchmark with the devtools window open, I've been getting 3.5GB heap-unclassified out of a 4GB heap in the main process. I ran with DMD enabled and managed to get up to 800MB heap-unclassified. One of the reports is pasted below. I will note that this allocation is nearly half slop.

Unreported {
  21,147 blocks in heap block record 2 of 7,706
  86,618,112 bytes (45,846,696 requested / 40,771,416 slop)
  Individual block sizes: 4,096 x 21,147
  7.12% of the heap (28.28% cumulative)
  10.33% of unreported (41.02% cumulative)
  Allocated at {
    #01: ??? (./Downloads/firefox/firefox)
    #02: moz_xmalloc (./Downloads/firefox/firefox)
    #03: mozilla::net::Http2Session::WriteSegmentsAgain(mozilla::net::nsAHttpSegmentWriter*, unsigned int, unsigned int*, bool*) (/home/sfink/Downloads/firefox/libxul.so)
    #04: mozilla::net::nsLoadGroup::AddRequest(nsIRequest*, nsISupports*) (/home/sfink/Downloads/firefox/libxul.so)
    #05: NS_IsSrcdocChannel(nsIChannel*) (/home/sfink/Downloads/firefox/libxul.so)
    #06: mozilla::MozPromise<bool, nsresult, false>::ThenInternal(already_AddRefed<mozilla::MozPromise<bool, nsresult, false>::ThenValueBase>, char const*) (/home/sfink/Downloads/firefox/libxul.so)
    #07: mozilla::MozPromise<bool, nsresult, false>::ThenValue<mozilla::net::HttpChannelParent::DoAsyncOpen(mozilla::ipc::URIParams const&, mozilla::Maybe<mozilla::ipc::URIParams> const&, mozilla::Maybe<mozilla::ipc::URIParams> const&, nsIReferrerInfo*, mozilla::Maybe<mozilla::ipc::URIParams> const&, mozilla::Maybe<mozilla::ipc::URIParams> const&, nsIPrincipal*, unsigned int const&, nsTArray<mozilla::net::RequestHeaderTuple> const&, nsTString<char> const&, mozilla::Maybe<mozilla::ipc::IPCStream> const&, bool const&, short const&, unsigned int const&, unsigned char const&, bool const&, unsigned int const&, bool const&, unsigned long const&, nsTString<char> const&, bool const&, nsTString<char> const&, bool const&, bool const&, bool const&, unsigned int const&, mozilla::Maybe<mozilla::net::LoadInfoArgs> const&, mozilla::Maybe<mozilla::net::nsHttpResponseHead> const&, nsTString<char> const&, unsigned int const&, unsigned long const&, mozilla::Maybe<mozilla::net::CorsPreflightArgs> const&, unsigned int const&, bool const&, bool const&, bool const&, bool const&, nsTString<char> const&, unsigned int const&, unsigned int const&, unsigned long const&, nsTString<char16_t> const&, unsigned long const&, nsTArray<mozilla::net::PreferredAlternativeDataTypeParams> const&, unsigned long const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, bool const&, mozilla::TimeStamp const&, bool const&)::$_34, mozilla::net::HttpChannelParent::DoAsyncOpen(mozilla::ipc::URIParams const&, mozilla::Maybe<mozilla::ipc::URIParams> const&, mozilla::Maybe<mozilla::ipc::URIParams> const&, nsIReferrerInfo*, mozilla::Maybe<mozilla::ipc::URIParams> const&, mozilla::Maybe<mozilla::ipc::URIParams> const&, nsIPrincipal*, unsigned int const&, nsTArray<mozilla::net::RequestHeaderTuple> const&, nsTString<char> const&, mozilla::Maybe<mozilla::ipc::IPCStream> const&, bool const&, short const&, unsigned int const&, unsigned char const&, bool const&, unsigned int const&, bool const&, unsigned long const&, nsTString<char> const&, bool const&, nsTString<char> const&, bool const&, bool const&, bool const&, unsigned int const&, mozilla::Maybe<mozilla::net::LoadInfoArgs> const&, mozilla::Maybe<mozilla::net::nsHttpResponseHead> const&, nsTString<char> const&, unsigned int const&, unsigned long const&, mozilla::Maybe<mozilla::net::CorsPreflightArgs> const&, unsigned int const&, bool const&, bool const&, bool const&, bool const&, nsTString<char> const&, unsigned int const&, unsigned int const&, unsigned long const&, nsTString<char16_t> const&, unsigned long const&, nsTArray<mozilla::net::PreferredAlternativeDataTypeParams> const&, unsigned long const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, mozilla::TimeStamp const&, bool const&, mozilla::TimeStamp const&, bool const&)::$_35>::~ThenValue() (/home/sfink/Downloads/firefox/libxul.so)
    #08: mozilla::net::UrlClassifierFeatureFlash::MaybeInitialize() (/home/sfink/Downloads/firefox/libxul.so)
  }
}

Comment 1

[Dragana Damjanovic [:dragana]]

Matt, is it connected to DocumentChannel?

Comment 2

[Matt Woodrow (:mattwoodrow)]

The stack there looks to have some incorrect symbols so it's hard to tell.

DocumentChannel should just be a placeholder until the real http channel is connected, so it's not obvious that leaks within http code would be related to DC.

Comment 3

[Junior [inactive]]

Align the priority with the meta bug 1549807

The stack in description seems not showing the whole stack
I see another hint fro the meta, but the volume is way lower

Unreported {
  1 block in heap block record 997 of 19,324
  32,768 bytes (32,768 requested / 0 slop)
  0.00% of the heap (41.30% cumulative)
  0.00% of unreported (97.21% cumulative)
  Allocated at {
    #01: nsSegmentedBuffer::AppendNewSegment() (in XUL) + 266
    #02: nsPipeOutputStream::WriteSegments(nsresult (*)(nsIOutputStream*, void*, char*, unsigned int, unsigned int, unsigned int*), void*, unsigned int, unsigned int*) (in XUL) + 1897
    #03: mozilla::net::nsHttpTransaction::WriteSegments(mozilla::net::nsAHttpSegmentWriter*, unsigned int, unsigned int*) (in XUL) + 1762
    #04: mozilla::net::Http2Stream::WriteSegments(mozilla::net::nsAHttpSegmentWriter*, unsigned int, unsigned int*) (in XUL) + 131
    #05: mozilla::net::Http2Session::WriteSegmentsAgain(mozilla::net::nsAHttpSegmentWriter*, unsigned int, unsigned int*, bool*) (in XUL) + 4739
    #06: mozilla::net::nsHttpConnection::OnInputStreamReady(nsIAsyncInputStream*) (in XUL) + 545
    #07: non-virtual thunk to mozilla::net::nsHttpConnection::OnInputStreamReady(nsIAsyncInputStream*) (in XUL) + 13
    #08: mozilla::net::nsSocketInputStream::OnSocketReady(nsresult) (in XUL) + 155
  }
}

And it's 80MB bytes, right?
Would you mind to change the summary if you agree, Steve?

Comment 4

[Junior [inactive]]

Sorry for the noise. I see you mention that.

Comment 5

[Steve Fink [:sfink] [:s:]]

Attachment: main process DMD report

In case it helps, here's the full DMD report. I don't know why the stack was mangled; this from a linux nightly, so perhaps it had updated on disk and had the wrong symbols? I don't know how that works.

Comment 6

[Steve Fink [:sfink] [:s:]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #2)

The stack there looks to have some incorrect symbols so it's hard to tell.

What symbols look incorrect?

I was trying to see how nsLoadGroup::AddRequest could possibly call Http2Session::WriteSegmentsAgain. I found a way, but it would require a ridiculous amount of inlining:

uint32 mozilla::net::nsLoadGroup::AddRequest(nsIRequest*, nsISupports*)
nsIRequestObserver.OnStartRequest:0
nsIStreamListener.OnStartRequest:0
mozilla::net::WebSocketChannel.OnStartRequest:0
uint32 mozilla::net::WebSocketChannel::OnStartRequest(nsIRequest*)
uint32 mozilla::net::WebSocketChannel::CallStartWebsocketData()
uint32 mozilla::net::WebSocketChannel::StartWebsocketData()
nsIAsyncInputStream.AsyncWait:0
mozilla::net::nsPreloadedStream.AsyncWait:0
uint32 mozilla::net::nsPreloadedStream::AsyncWait(nsIInputStreamCallback*, uint32, uint32, nsIEventTarget*)
nsIInputStreamCallback.OnInputStreamReady:0
mozilla::net::nsHttpConnection.OnInputStreamReady:0
uint32 mozilla::net::nsHttpConnection::OnInputStreamReady(nsIAsyncInputStream*)
uint32 mozilla::net::nsHttpConnection::OnSocketReadable()
mozilla::net::nsAHttpTransaction.WriteSegmentsAgain:0
mozilla::net::ASpdySession.WriteSegmentsAgain:0
mozilla::net::Http2Session.WriteSegmentsAgain:0
uint32 mozilla::net::Http2Session::WriteSegmentsAgain(mozilla::net::nsAHttpSegmentWriter*, uint32, uint32*, uint8*)

I have no idea whether that's even possible in practice. There are other ways too, but they require even deeper inlining:

uint32 mozilla::net::nsLoadGroup::AddRequest(nsIRequest*, nsISupports*)
nsIRequestObserver.OnStartRequest:0
nsIStreamListener.OnStartRequest:0
mozilla::net::InterceptedHttpChannel.OnStartRequest:0
uint32 mozilla::net::InterceptedHttpChannel::OnStartRequest(nsIRequest*)
uint32 nsInputStreamPump::PeekStream((void)(void*,uint8*,uint32)*, void*)
nsIAsyncInputStream.Available:0
mozilla::net::CacheFileInputStream.Available:0
uint32 mozilla::net::CacheFileInputStream::Available(uint64*)
void mozilla::net::CacheFileInputStream::EnsureCorrectChunk(uint8)
void mozilla::net::CacheFileInputStream::MaybeNotifyListener()
void mozilla::net::CacheFileInputStream::NotifyListener()
nsIInputStreamCallback.OnInputStreamReady:0
mozilla::net::nsHttpConnection.OnInputStreamReady:0
uint32 mozilla::net::nsHttpConnection::OnInputStreamReady(nsIAsyncInputStream*)
uint32 mozilla::net::nsHttpConnection::OnSocketReadable()
mozilla::net::nsAHttpTransaction.WriteSegmentsAgain:0
mozilla::net::ASpdySession.WriteSegmentsAgain:0
mozilla::net::Http2Session.WriteSegmentsAgain:0
uint32 mozilla::net::Http2Session::WriteSegmentsAgain(mozilla::net::nsAHttpSegmentWriter*, uint32, uint32*, uint8*)

(and there are others callchains possible as well, as long or longer.)

Comment 7

[Firefox Bug Husbandry Bot]

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is P3 (Backlog,) indicating it has been triaged, the bug's Severity is being updated to S3 (normal.)