1552326 - Implement Network.setRequestInterception

Status: RESOLVED WONTFIX

(Remote Protocol :: CDP, enhancement, P3)

Description

[Alexandre Poirot [:ochameau]]

See bug 1552325 comment 0.

This command enables the emission of Network.requestIntercepted event, which looks like this:

{"method":"Network.requestIntercepted","params":{"interceptionId":"id-1","request":{"url":"http://localhost:8889/wp-admin/post-new.php","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3494.0 Safari/537.36","Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","Cookie":"wordpress_23778236db82f19306f247e20a353a99=admin%7C1558202553%7CVuVC3Z18s9haDWOEEiOcN7Z4NWd3jdjorB94Vm6SErb%7C10b74bec70ab51a52c5d7b6c5dd4ea17fb80e6299d88b979e47b996c7fd24d3c; wordpress_test_cookie=WP+Cookie+check; wordpress_logged_in_23778236db82f19306f247e20a353a99=admin%7C1558202553%7CVuVC3Z18s9haDWOEEiOcN7Z4NWd3jdjorB94Vm6SErb%7C66d3f40ceebce50047aad36163b9a95b8348188efe559cb40f6971279586fa1e; wp-settings-1=editor%3Dtinymce; wp-settings-time-1=1558029753"},"initialPriority":"VeryHigh","referrerPolicy":"no-referrer-when-downgrade"},"frameId":"A966F77A1F3A97FF8CEEAD72EFE9B104","resourceType":"Document","isNavigationRequest":true}}

It also works in coordination with Network.continueInterceptedRequest request.

Comment 1

[Alexandre Poirot [:ochameau]]

This also spawn a Network.setCacheDisabled. But it may have changed in upstream version of puppeteer.

Comment 2

[Alexandre Poirot [:ochameau]]

These particular requests have been refactored in latest puppeteer.

Network.setRequestInterception is no longer called. And so Network.requestIntercepted is no longer listened and Network.continueInterceptedRequest is not longer called.

Network.setRequestInterception seems to be replaced by Fetch.enable and Fetch.disable.

You can see the diff between puppeteer 1.6.2 and master over here.

Note that Network.setCacheDisabled is still called in middle of the interception code.

I don't think there is any value in implementing the older API, so this is a bug where we might want to implement upstream version of puppeteer instead of the gutenber version (1.6.2).

Comment 3

[Julian Descottes [:jdescottes]]

Setting P3, cf previous comment.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Actually Network.setRequestInterception() is called for the experimental puppeteer-firefox API, which we aren't going to support given that this is based on the juggler code.

Also Gutenberg seems to still use this API, and we aren't sure when they will update to a more recent Puppeteer version.

Andreas, what is your take on that? Marking it invalid, or shall we continue leaving it open for now?

Comment 5

[Andreas Tolfsen ❲:ato❳]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #4)

Actually Network.setRequestInterception() is called for the
experimental puppeteer-firefox API, which we aren't going to
support given that this is based on the juggler code.

Based on ochameau’s comment #2, it used to be called by Gutenberg
through Puppeteer when Gutenberg was using an earlier version of
Puppeteer.

Juggler is unrelated, though it happens to expose an API that is
named similarly.

Also Gutenberg seems to still use this API, and we aren't sure
when they will update to a more recent Puppeteer version.

I thought they were now on an upgraded version of Puppeteer?

It may be that the list in https://wiki.mozilla.org/Remote/GutenbergCDPUsage
has not been updated since they upgraded, and we talked about this last week.

Andreas, what is your take on that? Marking it invalid, or shall
we continue leaving it open for now?

I think it’s probably right that we don’t want to implement this
if Puppeteer has moved to use something else.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I had another look and indeed Gutenberg only uses setRequestIntereption on the Page domain, which forwards everything to the NetworkManager class:

https://github.com/GoogleChrome/puppeteer/blob/master/lib/NetworkManager.js#L126-L150

As you can see the Fetch domain is used instead. As such we should indeed not track this for the alpha release, and maybe not at all for Puppeteer.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrey Lushnikov from bug 1535101 comment #0)

We implemented CDP's basic request interception using nsIRequest.suspend. Simple things work, but CDP allows to modify request method, request body, spoof the url and even fake the whole response. We didn't find a way to implement these with Gecko - are we missing something?

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We actually have to check again if Fetch needs the Network events to actually work.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

There is currently no priority for this bug. We should investigate when it becomes important.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Implementing Network interception is a tedious task that is currently done for Webdriver BiDi. We do not see that feature to be implemented for our CDP implementation.