1587960 - Simple listener webRequest.onBeforeRequest xmlhttprequest with filterResponseData results in page load failure

Status: VERIFIED FIXED

(WebExtensions :: Request Handling, defect, P1)

Description

[particlecore]

Attachment: test.zip

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0

Steps to reproduce:

Simple xmlhttprequest webRequestBlocking listener that does nothing but decode and encode the stream.

See attached extension example for testing.

Install it on Nightly (current version I tested on was 2019-10-10 also on the previous one 2019-10-09)
Open any YouTube video, for example: https://www.youtube.com/watch?v=YzbARjL1o-s
Select another video on the right side in order to navigate the page or simply click to go to the homepage
The top of the website shows a progress bar conveying the loading state for the navigation that was initiated

This is a Single Page Application style loading, it loads the next page via a JSON file and updates the DOM when it finishes receiving it.

Actual results:

The progress bar never reaches the end and the page never finishes loading. The next video might play, but that's the only thing that happens.

This problem was not happening until 3 days ago. I thought this was a problem with my code, but just removing the "filterResponseData" alone from the background script makes the page work normally again.

It was working perfectly fine before and modifications were being applied with success.

It feels like the website no longer receives the intercepted request after it is modified.

Expected results:

The page should finish loading, normal behavior should not be affected since there is nothing being modified on the request, the only thing being done is attaching a filterResponseData to the requestId.

Comment 1

[Alex Cornestean]

Hello,

I have managed to reproduce the issue on the latest Nightly (71.0a1/20191015213743) under Windows 10 Pro 64-bit and MacOS Catalina 10.15, with additional test on the latest Beta (70.0/20191014163058) and Release (69.0.3/20191009172106), where the issue does not occur.

With the extension loaded in the browser, the page fails to completely load, the progress bar not reaching the end. Removing the filterResponseData from the background script causes the page to function normally.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Comment 3

[Pascal Chevrel:pascalc (PTO until April 26)]

Jim, could you prioritize this bug wrt to our 71 release? I'd like to know if this is something we should track for a potential uplift in beta. Thanks.

Comment 4

[Shane Caraveo (:mixedpuppy)]

This is a regression on the current nightly, if we can get the regression window, it might be something solvable prior merging to beta.

Comment 5

[Alex Cornestean]

Hello,

I have performed a regression range on the issue and narrowed it down to this: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=720c1e5a8dd3f5bda4ae32137d1c624a1ad55301&tochange=be9a6289486a6f366e431782b84a0c0633f8fec2

Comment 6

[Shane Caraveo (:mixedpuppy)]

I've verified that flipping the pref dom.serviceWorkers.parent_intercept and restarting fixes/causes the problem.

Comment 7

[particlecore]

Whatever fix was implemented for this bug royally messed up the API, I have users suddenly not able to load the page at all after they've updated Firefox to 72.0a1 (2019-10-23)

The webpage is now showing just a Content Encoding Error everywhere:

https://github.com/ParticleCore/Iridium/issues/788

Comment 8

[Shane Caraveo (:mixedpuppy)]

(In reply to particlecore from comment #7)

Whatever fix was implemented for this bug

No patch is on this bug, if there were a fix it would be posted and the bug closed.

Comment 9

[particlecore]

(In reply to Shane Caraveo (:mixedpuppy) from comment #8)

(In reply to particlecore from comment #7)

Whatever fix was implemented for this bug

No patch is on this bug, if there were a fix it would be posted and the bug closed.

My apologies, I thought the last update meant it was already.

In that case I will open a new bug report for this problem.

Comment 10

[Miruna Curtean]

Hello Shane,
Could 1590898 be added as a blocker for this ticket as testing any potential fix for this would be more efficient after any work related to filterResponseData is done?

Comment 11

[Shane Caraveo (:mixedpuppy)]

That has the same reporter with almost an identical test. It would be easy to confirm (see comment 6) if it's the same issue, in which case I think it should be dup'd to this bug rather than blocking.

Comment 12

[Shane Caraveo (:mixedpuppy)]

1590898 is not related.

Comment 13

[Pascal Chevrel:pascalc (PTO until April 26)]

dom.serviceWorkers.parent_intercept is false on beta/release, marking as Disabled for 71.

Comment 14

[Perry Jiang [:perry] [no longer employee, use ni?]]

Attachment: Bug 1587960 - Replace StreamListenerParent's channel on internal redirects r=asuth

This is needed when to pass the right data to nested nsIStreamListeners.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8df9e44645e4
Replace StreamListenerParent's channel on internal redirects r=asuth,mattwoodrow,kmag

Comment 16

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/8df9e44645e4

Comment 17

[Miruna Curtean]

Verified fixed on Windows PRO 10 64-bit and on macOS Catalina 10.15 with FF Nightly 72.0a1 (20191111215252).
Tested with youtube / dailymotion pages and videos to verify that they load successfully when picking the next video.