1367203 - Most requests aren't being received by webRequest.onBeforeRequest listeners

Status: RESOLVED INVALID

(WebExtensions :: Request Handling, defect)

Description

[Wladimir Palant]

I can see this regression in current Firefox 55.0a1 nightly (2017-05-23) on Ubuntu, works fine in Firefox 53.0 on the other hand. This issue is reproducible regardless of whether E10S is enabled.

One way to see it is installing an Adblock Plus development build from https://downloads.adblockplus.org/devbuilds/adblockplusfirefox/. After installation you can open Developer Tools and go to the Adblock Plus panel - it will report all requests received via webRequest.onBeforeRequest which is considerably less than the ones visible in the Network panel. For example, when loading https://adblockplus.org/ I see only 9 requests, all originating from the YouTube frame - none of the requests from the top-level frame have been received.

Attaching the debugger to HttpObserverManager.observe() in WebRequest.jsm, I can see that it really only receives these few requests. However, when I register my own "http-on-modify-request" observer via scratchpad I get all requests. I don't really have an explanation for that (turning E10S off doesn't change anything).

Comment 1

[Andy McKay]

Setting to a P1, but Shane to investigate a bit first and check on the priority.

Comment 2

[Shane Caraveo (:mixedpuppy)]

I'm unable to reproduce this on 55.0a1 nightly (5/29).  I used this code to monitor from webextension:

browser.webRequest.onBeforeRequest.addListener(details => {
  dump(`**** beforerequest ${details.requestId} ${details.url}\n`);
}, { urls: ["<all_urls>"] });

and compared that output to the web console network tab.  I got exactly the same requests in both (something like 26 requests I think).  I tried this with tracking protection on and off to verify that was not interfering.  

I'd need a reliable STR that does not include a full blown addon to look further, but I really dont see how your own http-on-modify-request observer would see anything additional to the one in WebRequest.jsm, that doesn't make any sense.

Comment 3

[Wladimir Palant]

Ouch, my bad. Not sure what went wrong with my debugging but the real issue is bug 1369284.