986082 - add memory reporters for the SPDY zlib buffers

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[Nathan Froyd [:froydnj]]

These were taking up enough memory (megabytes for one or two websites) that I decided to split this out into its own bug.

Comment 1

[Nathan Froyd [:froydnj]]

Attachment: add memory reporters for SPDY zlib buffers

This patch is pretty straightforward: write memory allocation wrappers that
record how much they've allocated, register a new memory reporter, and use the
new wrappers in SpdySession* as appropriate.

Comment 2

[Patrick McManus [:mcmanus]]

Comment on attachment 8394318 [details] [diff] [review]
add memory reporters for SPDY zlib buffers

Review of attachment 8394318 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm; but I'll pass the review to nick.. as he probably wants to add something similar to the http/2 code. (the http/2 HPACK compression scheme uses considerably less memory though.)

Comment 3

[Nicholas Nethercote [inactive]]

Comment on attachment 8394318 [details] [diff] [review]
add memory reporters for SPDY zlib buffers

Review of attachment 8394318 [details] [diff] [review]:
-----------------------------------------------------------------

This is a good start, but it's not ambitious enough. We use zlib in several places, and so this has wider potential use. I can foresee wanting to have multiple ZlibReporters, e.g. we could use one in http://dxr.mozilla.org/mozilla-central/source/js/src/vm/Compression.cpp#14-39.

So to allow that, sAmount can't be static; change it to a normal member. And then move the new files somewhere more generic, probably xpcom/base/. I think that should work.

(This gets me thinking about the other numerous reporters that have |sAmount| in them. Seems like they could be converted to |mAmount| easily enough, and then the |hasRun| check in the constructor wouldn't be needed. I wonder if at some point in the past when reporters were structured differently having |sAmount| made things simpler. Not that it really matters.)

::: netwerk/protocol/http/nsHttpHandler.cpp
@@ +211,5 @@
>  
>      LOG(("Creating nsHttpHandler [this=%p].\n", this));
>  
> +    nsresult rv = RegisterStrongMemoryReporter(new ZlibReporter());
> +    printf("register result: %u\n", uint32_t(rv));

Remove this debugging printf, plz.

Comment 4

[u408661]

As Patrick said, I'll have to add something similar to the HPACK implementation for HTTP/2 (disabled by default for now, so not a pressing matter). Thanks!

Comment 5

[Nathan Froyd [:froydnj]]

(In reply to Nicholas Nethercote [:njn] from comment #3)
> This is a good start, but it's not ambitious enough. We use zlib in several
> places, and so this has wider potential use. I can foresee wanting to have
> multiple ZlibReporters, e.g. we could use one in
> http://dxr.mozilla.org/mozilla-central/source/js/src/vm/Compression.cpp#14-
> 39.
> 
> So to allow that, sAmount can't be static; change it to a normal member. And
> then move the new files somewhere more generic, probably xpcom/base/. I
> think that should work.

Hm, making it a normal member means that we'd have to have a reference to the memory reporter at the allocation site.  I think it's better to not require that, which will necessitate some trickery.

> (This gets me thinking about the other numerous reporters that have
> |sAmount| in them. Seems like they could be converted to |mAmount| easily
> enough, and then the |hasRun| check in the constructor wouldn't be needed. I
> wonder if at some point in the past when reporters were structured
> differently having |sAmount| made things simpler. Not that it really
> matters.)

Same concern here.

Comment 6

[Nicholas Nethercote [inactive]]

Comment on attachment 8394318 [details] [diff] [review]
add memory reporters for SPDY zlib buffers

Review of attachment 8394318 [details] [diff] [review]:
-----------------------------------------------------------------

Mmm, yes. You're right. In that case, would it make sense to rename it as SpdyZlibReporter?

Comment 7

[Nathan Froyd [:froydnj]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/2de3bb4941ed

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/2de3bb4941ed