1833656 - HAR multiple pages can miss pages if requests list is filtered

Status: RESOLVED FIXED

(DevTools :: Netmonitor, defect, P3)

Description

[Julian Descottes [:jdescottes]]

In order to reconstruct the page entries, we need to make sure we have all the navigation requests available to the HAR builder. However the DevTools HAR builder actually uses the list of displayed requests as the source for the HAR.

So for instance if you filter to display only images, the HAR builder will not have access to any of the navigation requests, and will then fail to find the accurate timings for the page navigations.

I am not sure allowing to use a filtered list of requests for HAR makes sense, but if we want to preserve the feature we have a few options:

1. Always send the full sorted requests list

This could be done from the connector (and therefore would have limited impact on the call sites), but the connector needs access to the state for this which is not guaranteed. Alternatively we can make this an explicit option, but this means a bigger impact on the netmonitor components which will now need to give access to the sorted requests list to any component which displays a context menu with HAR options.

2. Collect navigation requests on the side

The collector (or maybe another helper class, dedicated to collect har-specific data?) could watch for network resources an keep track of all requests which have isNavigationRequest set to true. Having a separate class would probably be the best because it starts to feel really hacky if we add more and more har-specific logic in the base collector.

Comment 1

[Julian Descottes [:jdescottes]]

Attachment: Bug 1833656 - [devtools] Support multiple pages in HAR with filtered requests

Depends on D174930

Comment 2

[Julian Descottes [:jdescottes]]

Attachment: Bug 1833656 - [devtools] Use real context menu item to save as HAR from har mochitests

Depends on D178336

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b3c9e8328a9f
[devtools] Support multiple pages in HAR with filtered requests r=bomsy,ochameau,devtools-reviewers
https://hg.mozilla.org/integration/autoland/rev/bbafcf23c9dd
[devtools] Use real context menu item to save as HAR from har mochitests r=bomsy,devtools-reviewers

Comment 4

[Serban Stanca [:SerbanS]]

Backed out for HAR related mochitests failures in browser_ext_devtools_network.js

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | browser/components/extensions/test/browser/browser_ext_devtools_network.js | HAR log should not be empty - Got +0, expected 1

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8fd82159e478
[devtools] Support multiple pages in HAR with filtered requests r=bomsy,ochameau,devtools-reviewers
https://hg.mozilla.org/integration/autoland/rev/25040b193885
[devtools] Use real context menu item to save as HAR from har mochitests r=bomsy,devtools-reviewers

Comment 6

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/8fd82159e478
https://hg.mozilla.org/mozilla-central/rev/25040b193885