1726362 - Saving some webpages fails when using strict tracking protection

Status: NEW

(Firefox :: File Handling, defect, P3)

Description

[[e-mail]]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0

Steps to reproduce:

Try to save webpage:
https://www.bd2020.com/jd/26885.htm

Right click -> Save Page As... (or Ctrl + S).

Actual results:

Fails to download webpage

Expected results:

Download webpage.

Comment 1

[[e-mail]]

Attachment: firefox-cannot-save-page-Failed.png

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Security' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 3

[Daniel Veditz [:dveditz]]

[NSFW warning: photo links "below the fold" have barely-clothed women in suggestive poses, ads have occasional cartoon nudity]

The site saved just fine for me on both Firefox 91 and Nightly 93.

Things to check:

1. disable all your addons and try again. If the page saves you can turn them back on one at a time to see which addon is causing the problem.
2. Open up the developer tools and see if there are any relevant error messages in the console.

Comment 4

[[e-mail]]

I think the issue may come from the adds. The first time I tried to save this page today worked, however the second time failed. Try several times to reload and save, eventually may trigger the issue:

Error: Promised response from onMessage listener went out of scope ExtensionMessagingService.js:89:34
Uncaught TypeError: e is undefined
t https://s.ssl.qhres2.com/ssl/ab77b6ea7f3fbf79.js:1
<anonymous> https://s.ssl.qhres2.com/ssl/ab77b6ea7f3fbf79.js:1
<anonymous> https://s.ssl.qhres2.com/ssl/ab77b6ea7f3fbf79.js:1
ab77b6ea7f3fbf79.js:1:74
t https://s.ssl.qhres2.com/ssl/ab77b6ea7f3fbf79.js:1
<anonymous> https://s.ssl.qhres2.com/ssl/ab77b6ea7f3fbf79.js:1
<anonymous> https://s.ssl.qhres2.com/ssl/ab77b6ea7f3fbf79.js:1

Comment 5

[[e-mail]]

Attachment: bd2020-error.png

Comment 6

[[e-mail]]

These are the warnings (several about cookies):

This page uses the non standard property “zoom”. Consider using calc() in the relevant property values, or using “transform” along with “transform-origin: 0 0”. 26885.htm
Some cookies are misusing the recommended “SameSite“ attribute 9
Cookie “Hm_lvt_59d9e8ec4b64f4d757c3a71905edaaf9” will be soon rejected because it has the “SameSite” attribute set to “None” or an invalid value, without the “secure” attribute. To know more about the “SameSite“ attribute, read https://developer.mozilla.org/docs/Web/HTTP/Headers/Set-Cookie/SameSite hm.js:1:559

Comment 7

[[e-mail]]

Attachment: ab77b6ea7f3fbf79.js

Comment 8

[[e-mail]]

• Browser console:

sendRemoveListener on closed conduit {036a55b4-5e72-4d05-a06c-cba2dfcc134a}.1511828488204 ConduitsChild.jsm:108
_send resource://gre/modules/ConduitsChild.jsm:108
_send self-hosted:1279
removeListener resource://gre/modules/ExtensionChild.jsm:663
removeListener resource://gre/modules/ExtensionChild.jsm:886
onChanged chrome://extensions/content/child/ext-storage.js:332
removeListener resource://gre/modules/ExtensionCommon.jsm:2534
revoke resource://gre/modules/ExtensionCommon.jsm:2556
close resource://gre/modules/ExtensionCommon.jsm:2561
unload resource://gre/modules/ExtensionCommon.jsm:922
close resource://gre/modules/ExtensionContent.jsm:934
destroyed resource://gre/modules/ExtensionContent.jsm:1012
observe resource://gre/modules/ExtensionContent.jsm:1030
hm.baidu.com : server does not support RFC 5746, see CVE-2009-3555 2
Error: Please use $(ref:runtime.getURL). whatfont_activate.js:2
<anonymous> moz-extension://6500f99b-7b4d-427f-b765-bf09e1122863/whatfont_activate.js:2
Error: Please use $(ref:runtime.getURL). 2 background.js:31
d moz-extension://df3aff4d-8ec3-46a0-b3bc-7e0a6e440579/background/background.js:31
No chrome package registered for chrome://thunder/content/thunder.png

• This is exactly when saving:

Error: Please use $(ref:runtime.getURL). background.js:31
d moz-extension://df3aff4d-8ec3-46a0-b3bc-7e0a6e440579/background/background.js:31
No chrome package registered for chrome://thunder/content/thunder.png

Comment 9

[[e-mail]]

These are the contents of moz-extension://df3aff4d-8ec3-46a0-b3bc-7e0a6e440579/

300: jar:file:///C:/Users/coteco/AppData/Roaming/Mozilla/Firefox/Profiles/vaardi3s.default/extensions/[e-mail]!/
200: filename content-length last-modified file-type
201: META-INF/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY
201: _locales/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY
201: background/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY
201: icons/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY
201: manifest.json 1178 Thu,%2029%20Nov%201979%2023:00:00%20GMT FILE
201: mozilla-recommendation.json 170 Thu,%2029%20Nov%201979%2023:00:00%20GMT FILE
201: options/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY
201: popup/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY
201: replaced/ 0 Mon,%2031%20Dec%201979%2023:00:00%20GMT DIRECTORY

Comment 10

[[e-mail]]

Attachment: firefox-cannot-save-page-Failed.png

Comment 11

[Pablo]

I was able to reproduce this using windows10 64bit and Ubuntu 20 64bit. mac 10.15 did not fail, not sure if it was because its random.
reproduced with Firefox Release 91.0.1, beta 92.0b7 and also Nightly 93.0a1

Comment 12

[:Gijs (he/him)]

Probably a dupe of bug 1668530 or bug 1445211?

Comment 13

[Marco Bonardo [:mak]]

In the network panel I see both some connections blocked by UO and some mixed content. Let's add dependencies and we can check back once those are fixed.

Comment 15

[Simona Badau, Desktop QA]

I've reproduced the download failing issue using the latest Nightly 97.0a1 on macOS Big Sur 11.6 also on:
https://www.dw.com/en/how-chinas-mines-rule-the-market-of-critical-raw-materials/a-57148375

Modifying the summary of the report to reflect this.
Thanks!

Comment 16

[deference]

As the URL above is NSFW, here are the ones I found (in the now duplicate bug). AFAIK, they're clean:
https://www.dw.com/en/how-chinas-mines-rule-the-market-of-critical-raw-materials/a-57148375
https://www.theepochtimes.com/new-us-senate-bill-seeks-to-boost-domestic-rare-earth-production-wean-off-chinese-imports_2999724.html
https://www.zdnet.com/article/us-president-biden-signs-law-to-ban-huawei-and-zte-from-receiving-fcc-licences/
https://www.theverge.com/2021/6/3/22516719/biden-executive-order-american-investment-chinese-firms-huawei-zte

Additionally, this is not intermittent for me. I can save the webpages multiple times over and over without success.

Comment 17

[:Gijs (he/him)]

(In reply to deference from comment #16)

As the URL above is NSFW, here are the ones I found (in the now duplicate bug). AFAIK, they're clean:
https://www.dw.com/en/how-chinas-mines-rule-the-market-of-critical-raw-materials/a-57148375

WFM.

Can you try in a clean Firefox profile? I expect this is still related to either ads / tracking protection preferences, mixed content (but then I'd expect it to also fail for me) or add-ons that block bits of the page, as pointed out in comment #12 / comment 13.

Comment 18

[deference]

(In reply to :Gijs (he/him) from comment #17)

(In reply to deference from comment #16)

As the URL above is NSFW, here are the ones I found (in the now duplicate bug). AFAIK, they're clean:
https://www.dw.com/en/how-chinas-mines-rule-the-market-of-critical-raw-materials/a-57148375

WFM.

Can you try in a clean Firefox profile? I expect this is still related to either ads / tracking protection preferences, mixed content (but then I'd expect it to also fail for me) or add-ons that block bits of the page, as pointed out in comment #12 / comment 13.

As I already explained in my original report, this has nothing to do with add-ons. I tested it.
I'm now testing with a completely fresh config. The setting that affects these downloads is the "Tracking content in all windows" setting. It is set/reached from Preferences-> Privacy and Security-> (Strict|Custom).

Now personally, if an add-on or FF itself forbids access to a resource, I should think that the "file downloader" part of FF would respect that as it is the will of the user of the user agent.

Comment 19

[:Gijs (he/him)]

Paul, do you know if the tracking protection code identifies that that is what is canceling requests, so that it can be identified and ignored (rather than, say, network failure!)? (see bug 1604618 for the infrastructure addition)

Comment 20

[Paul Zühlcke [:pbz]]

The network channel is cancelled by the URLClassifier with an anti-tracking specific error code:

https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/netwerk/url-classifier/UrlClassifierFeatureTrackingProtection.cpp#180
https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/netwerk/protocol/http/HttpBaseChannel.cpp#5466

Looks like this is passed on via nsIRequest, which we could check.

As for hooking into the classifier externally and unblocking the request, you could maybe do it like here: https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/browser/extensions/webcompat/experiment-apis/trackingProtection.js#72

Passing the NI to Dimi who is more familiar with our tracker blocking integration.

Comment 21

[:Gijs (he/him)]

(In reply to Paul Zühlcke [:pbz] from comment #20)

The network channel is cancelled by the URLClassifier with an anti-tracking specific error code:

https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/netwerk/url-classifier/UrlClassifierFeatureTrackingProtection.cpp#180
https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/netwerk/protocol/http/HttpBaseChannel.cpp#5466

Looks like this is passed on via nsIRequest, which we could check.

Thanks.

As for hooking into the classifier externally and unblocking the request, you could maybe do it like here: https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/browser/extensions/webcompat/experiment-apis/trackingProtection.js#72

Passing the NI to Dimi who is more familiar with our tracker blocking integration.

To be clear, I don't think we want to do this - I think the tracker script or image should then not be downloaded, and the code at https://searchfox.org/mozilla-central/rev/5d2b9e940ca09bd1cbc15aa681f69424cde8904c/dom/webbrowserpersist/nsWebBrowserPersist.cpp#892-894 and friends will need teaching that some failures should be deemed non-fatal for the overall persisting-of-an-HTML-document process (and the rest of the code will need to deal with that somehow, ie in how the resulting references in the document get rewritten, as they will continue to fail once opening the saved document).

Comment 22

[Dimi Lee [:dimi]]

(In reply to :Gijs (he/him) from comment #21)

(In reply to Paul Zühlcke [:pbz] from comment #20)

The network channel is cancelled by the URLClassifier with an anti-tracking specific error code:

Looks like this is passed on via nsIRequest, which we could check.

Thanks.

So other than NS_ERROR_TRACKING_URI error as Paul has pointed out, there are three more cases that we might want to consider:

• NS_ERROR_CRYPTOMINING_URI
• NS_ERROR_FINGERPRINTING_URI
• NS_ERROR_SOCIALTRACKING_URI

And besides getting from status from nsIRequest, an alternative way is getting the information from requestBlockingReason in nsILoadInfo
https://searchfox.org/mozilla-central/rev/c3cbf7e56630d12443459a6bd7938358c231ce3b/netwerk/base/nsILoadInfo.idl#1340

You can find the definition here:
https://searchfox.org/mozilla-central/rev/c3cbf7e56630d12443459a6bd7938358c231ce3b/netwerk/base/nsILoadInfo.idl#1312-1314
https://searchfox.org/mozilla-central/rev/c3cbf7e56630d12443459a6bd7938358c231ce3b/netwerk/base/nsILoadInfo.idl#1309