1575536 - Network monitor janks the parent process when a page fetches big files

Status: NEW

(DevTools :: Netmonitor, defect, P2)

Description

[Florian Quèze [:florian]]

See this profile: https://perfht.ml/30qrXVU

The problem is at these lines https://searchfox.org/mozilla-central/rev/03853a6e87c4a9405fce1de49e5d03b9e7a7a274/devtools/server/actors/network-monitor/network-response-listener.js#189,198 that process the whole file instead of just the beginning.

Comment 1

[:Harald Kirschner :digitarald]

P2, as it's highly actionable.

Comment 2

[Jan Honza Odvarko [:Honza] (always need-info? me)]

The problem is that the implementation of onDataAvailable must ready exactly count bytes of data before returning
See MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/NsIStreamListener

There is apparently no way to stop reading the stream when reaching the limit from prefs.

@kmaglione: not sure who to ask, but is there any way to skip reading the rest of the stream (nsIStreamListener) to get some performance win here? Or is there anyone else I could ask?

The NetworkHelper.convertToUnicode( can be optimized since don't have to convert the entire data to unicode, just the part that doesn't exceeds the limit.

Honza

Comment 3

[Kris Maglione [:kmag]]

So... there are a few things you can do here.

For a start, NetUtil.readInputStreamToString is very expensive. readInputStream would be much cheaper.

Beyond that, given the input stream tee approach you currently use, you can at least in principal just stop reading data, and the input stream tee will just give up writing data to the pipe once its buffer fills up.

But, really, the best solution is probably to stop using an input stream tee altogether, call channel.setNewListener with your response listener directly, have your onStartRequest/onDataAvailable/onStopRequest call the same methods on the original listener, and just pass on the original stream from onDataAvailable rather than reading from it yourself and making a copy once you've stopped caring about data.

If you go that route, though, you're really going to want the listener to be implemented in C++ so that it can support off-main-thread delivery. I'd lean towards just implementing the whole thing in C++, but you could of course just implement a stub that proxies the OnDataAvailable calls to the main thread and leaves most of the rest of the code as JS.

Comment 4

[Brad Werth [:bradwerth]]

I'll attempt to implement Kris's ideas in Comment 3.