1460494 - Network monitor header pane only works upon download completion

Status: NEW

(DevTools :: Netmonitor, defect, P3)

Description

[Craig Howard]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/65.0.3325.181 Chrome/65.0.3325.181 Safari/537.36

Steps to reproduce:

1. Open developer tools, switch to the network monitor
2. Browse to a page with a video (ie https://en.wikipedia.org/wiki/Big_Buck_Bunny)
3. Switch to the "All" tab
4. Start playing the video
5. Click on the Big_Buck_Bunny at the bottom of the list
6. Right click on Big_Buck_Bunny -> Copy -> Copy as CURL



Actual results:

After 5, the headers pane is empty
After 6, the headers pane is populated


Expected results:

After 5, the headers pane should be populated

Comment 1

[Craig Howard]

This is a branch of bug 1420513

Comment 2

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Thanks for the report!
Honza

Comment 3

[Robert Xiao]

UA doesn't look right - that's a Chrome UA. Could you post the UA or nightly build that you were using?

Comment 4

[Robert Xiao]

Note that this issue extends to the Cookies, Params, Timings and Security panes - basically, with an unfinished request, the only pane that should be blank is Response. Even after "Copy as cURL" (which now forces the response headers to be loaded via "this.props.connector.requestData(id, "requestHeaders");"), only the Headers pane is populated; Cookies remains blank.

In general I suspect what needs to happen is that we need to add "requestData" calls to each panel render function, but I don't know if this would affect performance (i.e. how often render() gets called). I'm also not sure why the request headers, cookies and post data aren't immediately available for every request in the net monitor - under what circumstances can you have a request in the net monitor without request information?

Maybe we need some kind of two-phase loading of the request data, where we populate request props as soon as the request hits the net monitor, and then populate response props when the request completes. But, I'm not familiar enough with the netmonitor codebase to know if this is sound or easily doable - Honza, comments?

Comment 5

[Craig Howard]

Ah, I posted the bug report using Chrome.  The User-Agent was auto-added.  Bug is present in both Firefox 59 and 62.0a1 (2018-05-09) (64-bit).  That's a user agent of:

Mozilla/5.0 (X11; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0

I think the key observation is that there are a few interesting events in the life-cycle of an HTTP request/response (plus DNS/TCP/QUIC/TLS events).

1. HTTP request created
2. HTTP request headers sent
3. HTTP request body sent
4. HTTP response headers received
5. HTTP response body received

As each of these things happen, ideally, there'd be more data immediately available in the net monitor.  My particular case calls out a lack of #4.  I haven't tested the rest.  I doubt (although I know little to nothing about the Firefox internals) it's viable to do a two-phase load, because the first load would be racing with the HTTP response header parsing.