1373776 - chrome.webRequest.onHeadersReceived / window.unload interaction bug

Status: RESOLVED WONTFIX

(WebExtensions :: Request Handling, defect)

Description

[Alexei]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36

Steps to reproduce:

There is some sort of bug in the interaction between a webpage with an unload listener that performs web requests, and WebExtensions add-on that uses chrome.webRequest.onHeadersReceived in Firefox 54 (maybe other versions too).

To reproduce, install the minimal testing add-on from https://github.com/EFForg/privacybadger/files/1079120/firefox_webext_onunload_bug_demo.zip and either set up and visit the website in https://github.com/adimascio/testff.git or visit an affected site in the wild, https://www.logilab.org/, and click on a link in the page.

Please see https://github.com/EFForg/privacybadger/issues/1438 for the original Privacy Badger bug report.


Actual results:

When clicking a link on a website with an unload listener that performs web requests, resources seem to get fetched and the browser's address bar gets updated, but the page does not get rendered; the old page is still there.


Expected results:

The browser should properly render the newly-navigated-to page.

Comment 1

[Alexei]

Hi, are you able to reproduce? Some Privacy Badger users are affected by this bug.

Comment 2

[Shane Caraveo (:mixedpuppy)]

I'm unable to reproduce this on either osx or linux on the current nightly, or on osx in current release.  I used the test site from github.

Comment 3

[Alexei]

Sorry, I think you're missing a step or two.

How to reproduce:

1. On Ubuntu 14.04, downloaded and unpacked firefox-54.0.1.tar.bz2
2. Downloaded and unpacked https://github.com/EFForg/privacybadger/files/1079120/firefox_webext_onunload_bug_demo.zip
3. Loaded the add-on from about:debugging.
4. Visited https://www.logilab.org/
5. Clicked on the first category link in the page (Artificial Intelligence, https://www.logilab.org/folder/847).
6. The URL bar updated, but the page didn't. Reloading made the page update.

Comment 4

[Alexei]

(firefox-54.0.1.tar.bz2 being the stable release today)

7. After reproducing, disable the add-on in about:addons > Extensions.
8. Visit https://www.logilab.org/ again.
9. Click on Artificial Intelligence.
10. Watch the URL bar and the page get correctly updated.

Then take a look at the contents of the add-on: it doesn't do much but it causes the issue to surface.

Comment 5

[Alexei]

While the above issue seems to be new to Firefox 54, I just got another user bug report from somebody using the 52.2 ESR release. The user confirmed that they can reproduce the issue with the same firefox_webext_onunload_bug_demo.zip test add-on alone. Their issue description follows:

"When I try to login to my Verizon FiOS account everything is good up and until the final sign in page, https://www.verizon.com/foryourhome/myaccount/pr/dashboard/details, which won’t load fully. In order to continue, I have to reload the page [...] and then I can get in. I have tried disabling [Privacy Badger] for Verizon for each stage of the login procedure, and then whitelisting everything Verizon I can find, but none of that helps. Appears to be something intrinsic to PB itself, since the only remedy to get into my account without reloading that hanging page is to completely disable PB. Wth PB completely disabled, no problem."

Comment 6

[Alexei]

As you can see, this Firefox bug is particularly frustrating, as the usual way of working around site breakages (telling the add-on to permit everything on the page) does not work. As you can see from the test add-on, just listening to received headers is enough to trigger the bug every time:

chrome.webRequest.onHeadersReceived.addListener(function (/*details*/) {
  return {};
}, {
  urls: ["http://*/*", "https://*/*"]
}, [
  "blocking",
  "responseHeaders",
]);

Comment 7

[Shane Caraveo (:mixedpuppy)]

(In reply to Alexei from comment #3)
> Sorry, I think you're missing a step or two.

nope, still cant reproduce it.  Not saying it's not happening, just not reproducible here.  Is this STR working for you on a fresh profile?

Comment 8

[Alexei]

What does STR mean? I have been testing with fresh, temporary profiles (firefox -profile "$PROFILEDIR" -no-remote -new-instance). I'm on Ubuntu 14.04, the GitHub reporter is on Debian Stretch, the ESR user is on macOS 10.11.6.

Comment 9

[Shane Caraveo (:mixedpuppy)]

(In reply to Alexei from comment #8)
> What does STR mean? 

Sorry about that...Steps To Reproduce, which you provided in comment 3.

> I have been testing with fresh, temporary profiles
> (firefox -profile "$PROFILEDIR" -no-remote -new-instance). I'm on Ubuntu
> 14.04, the GitHub reporter is on Debian Stretch, the ESR user is on macOS
> 10.11.6.

I'm taking a guess is there some timing issue that is different between us.  IIRC, window.unload does not promise to wait for anything (ie. shouldn't be doing async calls there).  So doing a blocking webrequest listener on a request done in unload feels like it could be problematic, we just need to figure out how to reproduce it here to have a good idea what the issue is.

Comment 10

[Alexei]

Attachment: Reproduction video of bug on Ubuntu 17.04

A colleague of mine was able to reproduce on a clean install of Ubuntu
16.04 with Firefox 54, on Ubuntu 17.04 with a new profile of Firefox 54, and Windows 10 (same). Attaching a video from Ubuntu 17.04.

Comment 11

[Alexei]

Could you ask your colleagues to try this?

As far as the demo add-on goes, if I recall correctly, the bit in the onHeadersReceived listener that makes the bug appear is that the listener is declared as "blocking". It doesn't seem that the "requestHeaders" capability, or the callback's return value make a difference.

Comment 12

[Alexei]

Here is another report on Privacy Badger's GitHub, this one concerning eBay: https://github.com/EFForg/privacybadger/issues/1530

Comment 13

[Alexei]

Also, are you able to reproduce with Privacy Badger (https://www.eff.org/privacybadger) instead of my demo add-on?

Comment 14

[Alexei]

Several more user reports caused by this Firefox bug, all concerning eBay:

- https://github.com/EFForg/privacybadger/issues/1295#issuecomment-292648502
- https://github.com/EFForg/privacybadger/issues/1295#issuecomment-298888658
- https://github.com/EFForg/privacybadger/issues/1295#issuecomment-320287804

Can I do anything to help move this bug report along towards a solution?

Comment 15

[Andy McKay]

Maybe someone in QA could help Shane reproduce this? At the moment I think he's struggling to reproduce.

Comment 16

[Shane Caraveo (:mixedpuppy)]

Luca also tried to reproduce it and could not.  As I understand, he tried 54 through nightly.

Comment 17

[Shane Caraveo (:mixedpuppy)]

Alexi, can you verify your repro on 55,56 and 57?

Comment 18

[Alexei]

Using steps in https://bugzilla.mozilla.org/show_bug.cgi?id=1373776#c3.

Stable (firefox-55.0.2):
Unable to reproduce.

Beta (firefox-56.0b3):
Unable to reproduce.

Nightly (firefox-57.0a1.en-US.linux-x86_64):
Unable to reproduce.

Older versions:

54.0.1:
Reproduced.

Firefox 55.0b5:
Reproduced.

It looks like this (whatever it was) was fixed some time between Firefox 55 went from Beta to Stable.

Sorry for the delay; I was on vacation.

Comment 19

[Shane Caraveo (:mixedpuppy)]

Thanks Alexei!

Comment 20

[Graham Perrin]

Firefox ESR 52.8.1 (64-bit) on pre-release build 16G1408 on Mac OS X 10.12.6 (macOS Sierra) with Privacy Badger 2018.5.10 alone, no other extension.

<http://logilab.org/> is: 

- OK with multi-process Firefox ESR
- bugged with single-process Firefox ESR. 

That's for reference only; I don't expect a fix for any single-process use case. 

----

On the Waterfox 56.2.0 side of things, from <https://github.com/MrAlex94/Waterfox/issues/580#issuecomment-395860400>: 

> I assume that a (Mozilla) commit was made somewhere between Firefox 56.0.2 (upon which Waterfox 56.0 was based) and Firefox 60.