Closed
Bug 1484713
Opened 7 years ago
Closed 7 years ago
Tracking Protection: Massively increased page load time on spiegel.de
Categories
(Core :: Networking: HTTP, defect)
Tracking
()
VERIFIED
FIXED
mozilla63
| Tracking | Status | |
|---|---|---|
| firefox-esr52 | --- | unaffected |
| firefox-esr60 | --- | unaffected |
| firefox61 | --- | unaffected |
| firefox62 | --- | unaffected |
| firefox63 | + | verified |
People
(Reporter: jan, Assigned: ehsan.akhgari)
References
()
Details
(Keywords: nightly-community, regression)
Attachments
(2 files)
It's Germany's most popular news website.
1. Install uBlock Origin
2. Navigate on spiegel.de: Reload, click on an article.
3. The regression: If page loading takes a lot time, but you already see the background color and are even able to hover things.
Doesn't happen without Tracking Protection.
mozregression --launch 2018-08-01 --pref privacy.trackingprotection.enabled:true -a https://addons.mozilla.org/firefox/addon/ublock-origin/ -a http://www.spiegel.de/
> 11:38.48 INFO: Last good revision: e5d982e6568b576838f9674c48b9603651124e56
> 11:38.48 INFO: First bad revision: acf7da1e9687a774c9a091b18cc75c49717ed4aa
> 11:38.48 INFO: Pushlog:
> https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=e5d982e6568b576838f9674c48b9603651124e56&tochange=acf7da1e9687a774c9a091b18cc75c49717ed4aa
> acf7da1e9687 Ehsan Akhgari — Bug 1478539 - Part 3: Remove some dead code; r=nwgh
> 237c1635b0e9 Ehsan Akhgari — Bug 1478539 - Part 2: Add a test case to ensure that we don't send existing cookies in restricted third-party storage contexts; r=baku
> 434ed946ad61 Ehsan Akhgari — Bug 1478539 - Part 1: Add the Cookie header to HTTP requests only after a potential classification has been completed on the channel to ensure the tracking state is up to date; r=nwgh
|
Reporter | |
Comment 1•7 years ago
|
||
(With TP but without uBlock Origin you are not able to circumvent their Adblock overlay. That has been the case for a long time.)
The problem does not happen when using uBlock Origin without TP. Not even if you completely block cookies & site data.
|
|
Reporter | |
Comment 2•7 years ago
|
||
Sorry, copied the wrong command. It was:
mozregression --good 2018-08-01 --bad 2018-08-19 --pref privacy.trackingprotection.enabled:true -a https://addons.mozilla.org/firefox/addon/ublock-origin/ -a http://www.spiegel.de/
|
|
Reporter | |
Comment 3•7 years ago
|
||
The problem is a script adding a class "async-hide" to <html>.
Without uBlock and with blocked cookies it is removed after a short moment.
(Thunderbird/RSS: javascript.enable;false is unaffected from this. Only videos don't work then.)
|
Assignee | |
Comment 4•7 years ago
|
||
I can reproduce!
Assignee: nobody → ehsan
tracking-firefox63:
--- → ?
|
|
Assignee | |
Updated•7 years ago
|
Flags: needinfo?(ehsan)
|
|
Reporter | |
Comment 5•7 years ago
|
||
(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #3)
> The problem is a script adding a class "async-hide" to <html>.
https://www.kino.de/ seems to be affected as well.
|
||
Comment 6•7 years ago
|
||
Isn't this by chance a duplicate of bug 1420885?
|
|
||
Comment 7•7 years ago
|
||
(In reply to Honza Bambas (:mayhemer) from comment #6)
> Isn't this by chance a duplicate of bug 1420885?
Note that I was somewhat hoping this would be fixed by fastblock=on and tailing=off.
| Comment hidden (typo) |
|
|
Reporter | |
Comment 9•7 years ago
|
||
Could you introduce an unbreak feature (required for sure at some point) and just block that A/B testing script?
Any non-Opt-In analytics illegitimately tracks behavior from TP's point of view. As TP usually implies DNT=1 in Firefox:
From bug 1462851:
DNT=1 legally forbids tracking activities related to advertising, market research and personalization (§15 (3) german TMG).
DE: https://www.gesetze-im-internet.de/tmg/__15.html
> Der Diensteanbieter darf für Zwecke der Werbung, der Marktforschung oder zur bedarfsgerechten Gestaltung der Telemedien Nutzungsprofile bei Verwendung von Pseudonymen erstellen, sofern der Nutzer dem nicht widerspricht. Der Diensteanbieter hat den Nutzer auf sein Widerspruchsrecht im Rahmen der Unterrichtung nach § 13 Abs. 1 hinzuweisen. Diese Nutzungsprofile dürfen nicht mit Daten über den Träger des Pseudonyms zusammengeführt werden.
EN: https://www.huntonprivacyblog.com/wp-content/uploads/sites/28/2016/02/Telemedia_Act__TMA_.pdf
> For the purposes of advertising, market research or in order to design the telemedia in a needs-based manner, the service provider may produce profiles of usage based on pseudonyms to the extent that the recipient of the service does not object to this. The service provider must refer the recipient of the service to his right of refusal pursuant to Sub-section 13 No. 1. These profiles of usage must not be collated with data on the bearer of the pseudonym.
For sure, the correct way would be to file a complaint at the state's data protection office
(e.g. https://www.lfd.niedersachsen.de/themen/internet/telemedien/telemedien-im-internet-94379.html).
|
|
Assignee | |
Comment 10•7 years ago
|
||
(In reply to Honza Bambas (:mayhemer) from comment #7)
> (In reply to Honza Bambas (:mayhemer) from comment #6)
> > Isn't this by chance a duplicate of bug 1420885?
>
> Note that I was somewhat hoping this would be fixed by fastblock=on and
> tailing=off.
No, doesn't seem like it's a dupe. It happens with network.http.tailing.enabled set to false as well. Also, the bug certainly goes away when I back out https://hg.mozilla.org/mozilla-central/rev/434ed946ad61 locally. The bug has also nothing to do with uBlock Origin being present, it happens without it too.
The problem is, I have zero theories as to why that change could have made an impact like this. :-(
|
|
Assignee | |
Comment 11•7 years ago
|
||
HTTP log captured from a test session in a private window...
|
|
||
Comment 12•7 years ago
|
||
(In reply to :Ehsan Akhgari from comment #10)
> (In reply to Honza Bambas (:mayhemer) from comment #7)
> > (In reply to Honza Bambas (:mayhemer) from comment #6)
> > > Isn't this by chance a duplicate of bug 1420885?
> >
> > Note that I was somewhat hoping this would be fixed by fastblock=on and
> > tailing=off.
>
> No, doesn't seem like it's a dupe. It happens with
> network.http.tailing.enabled set to false as well.
if TP is on you will get the same behavior.
> Also, the bug certainly
> goes away when I back out
> https://hg.mozilla.org/mozilla-central/rev/434ed946ad61 locally.
interesting, no idea how that could be related. only some extension interference cold relate to this change.
> The bug
> has also nothing to do with uBlock Origin being present, it happens without
> it too.
>
> The problem is, I have zero theories as to why that change could have made
> an impact like this. :-(
either analytics.js or the dynamically loaded experiment script (probably https://www.google-analytics.com/collect) are blocked. I don't see even the requests (channels) made for any of them in the log you provide. maybe you cut the log too early?
|
|
Assignee | |
Comment 13•7 years ago
|
||
(In reply to Honza Bambas (:mayhemer) from comment #12)
> (In reply to :Ehsan Akhgari from comment #10)
> > (In reply to Honza Bambas (:mayhemer) from comment #7)
> > > (In reply to Honza Bambas (:mayhemer) from comment #6)
> > > > Isn't this by chance a duplicate of bug 1420885?
> > >
> > > Note that I was somewhat hoping this would be fixed by fastblock=on and
> > > tailing=off.
> >
> > No, doesn't seem like it's a dupe. It happens with
> > network.http.tailing.enabled set to false as well.
>
> if TP is on you will get the same behavior.
Interesting, I didn't know that!
> > Also, the bug certainly
> > goes away when I back out
> > https://hg.mozilla.org/mozilla-central/rev/434ed946ad61 locally.
>
> interesting, no idea how that could be related. only some extension
> interference cold relate to this change.
I've verified that the bug can be reproduced without any extensions installed. My first guess was also some extension interference (since my patch had something to do with http-on-modify-request...)
> > The bug
> > has also nothing to do with uBlock Origin being present, it happens without
> > it too.
> >
> > The problem is, I have zero theories as to why that change could have made
> > an impact like this. :-(
>
> either analytics.js or the dynamically loaded experiment script (probably
> https://www.google-analytics.com/collect) are blocked. I don't see even the
> requests (channels) made for any of them in the log you provide. maybe you
> cut the log too early?
Perhaps. This log includes google-analytics.com, is it more useful?
Flags: needinfo?(ehsan)
|
|
||
Comment 14•7 years ago
|
||
(In reply to :Ehsan Akhgari from comment #13)
> Created attachment 9002863 [details]
> HTTP log (bzip2 compressed)
> > interesting, no idea how that could be related. only some extension
> > interference cold relate to this change.
>
> I've verified that the bug can be reproduced without any extensions
> installed. My first guess was also some extension interference (since my
> patch had something to do with http-on-modify-request...)
could have an influence on channel annotation? or its timing related to blocking? didn't the patch fixed a bug around TP?
> Perhaps. This log includes google-analytics.com, is it more useful?
sorry, it doesn't :/ no request with 'analytics.js' in the URL.
|
|
||
Comment 15•7 years ago
|
||
Based on that (no analytics.js request in the logs) this seems to be a bug coming from something else than TP, since TP works (blocks) on existing channels only.
I know script loader is somehow marking URLs as trackers as well. is there some code in the script loader that could potentially block requests from even being created? I'm not aware of anything, but that is the next place I'd look at.
|
|
Assignee | |
Comment 16•7 years ago
|
||
(In reply to Honza Bambas (:mayhemer) from comment #15)
> Based on that (no analytics.js request in the logs) this seems to be a bug
> coming from something else than TP, since TP works (blocks) on existing
> channels only.
OK, good to know. Thanks! (Pretty sure google-analytics.com is blocked by TP, BTW.)
> I know script loader is somehow marking URLs as trackers as well. is there
> some code in the script loader that could potentially block requests from
> even being created? I'm not aware of anything, but that is the next place
> I'd look at.
I'm quite certain that's not the case. (I wrote the code that marks the URLs as trackers in script loader; there's no other magic there.)
|
|
Assignee | |
Comment 17•7 years ago
|
||
(In reply to Honza Bambas (:mayhemer) from comment #14)
> (In reply to :Ehsan Akhgari from comment #13)
> > Created attachment 9002863 [details]
> > HTTP log (bzip2 compressed)
> > > interesting, no idea how that could be related. only some extension
> > > interference cold relate to this change.
> >
> > I've verified that the bug can be reproduced without any extensions
> > installed. My first guess was also some extension interference (since my
> > patch had something to do with http-on-modify-request...)
>
> could have an influence on channel annotation?
The effect of the patch is that channel annotations now happen before AddCookiesToRequest() is called. The reason why I had to make the change was that AddCookiesToRequest() needed to know whether the channel was loading a tracking resource or not, so the classifier needed to have run before it got called.
> or its timing related to
> blocking?
It certainly does change timing! The problem is that I don't have a good understanding of all of the asynchronous semantics of HTTP channels to be able to form a theory as to how the change could have made this impact... :-/
> didn't the patch fixed a bug around TP?
Not at all. The bug is about the new cookie policy being developed in bug 1473978.
> > Perhaps. This log includes google-analytics.com, is it more useful?
>
> sorry, it doesn't :/ no request with 'analytics.js' in the URL.
I think the reason is that tracking protection blocks Google Analytics. There is no reason to expect analytics.js to show up in the log. ;-)
|
|
Assignee | |
Comment 18•7 years ago
|
||
Here is some more information. The "async-hide" class comes from Google Optimize: <https://developers.google.com/optimize/>.
See this code snippet:
<style>.async-hide { opacity: 0 !important} </style>
<script>(function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;h.start=1*new Date;
h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')};
(a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;
})(window,document.documentElement,'async-hide','dataLayer',4000,
{'GTM-XXXXXX':true});</script>
If you decode it, it sets the async-hide class on the document element, and sets up a timeout to clear that from the class name in 4 seconds. You're supposed to pass that "GTM-XXXXXX" ID to things like Google Analytics or Google Tag Manager so that when they load they unhide the page (like https://gist.github.com/simondahla/3299703295771ba69a8761892a6e01b2).
kino.de uses GA, and spiegel.de uses GTM.
|
|
Assignee | |
Comment 19•7 years ago
|
||
I thought of one change in behavior that bug 1478539 has caused here which I filed as bug 1485182. I'm fairly certain that this bug is a duplicate of that bug in fact. Here is why.
If you test this with Firefox release in a new profile, both spiegel.de and kino.de demonstrate the visible load delay, which is I think caused by the code in comment 18.
However, previously, if you had uBlock Origin installed, for some reason this delayed behavior wouldn't happen. I _think_ the reason is that uBlock Origin actually doesn't block these scripts, and provides a shim replacement instead. However when bug 1478539 landed, tracking protection started to cancel the channels before uBlock Origin had a chance to examine them, and as a result the same behavior that was visible without uBlock Origin started to appear.
With the patch on bug 1485182 applied, on a new profile *without* uBlock Origin, the visible delay is present, but *with* uBlock Origin, there is no visible delay.
I decided to file a separate bug to give Jan a chance to test this out in case there is more than one behavior change hiding here...
Depends on: 1485182
|
|
||
Comment 20•7 years ago
|
||
(In reply to :Ehsan Akhgari from comment #17)
> > sorry, it doesn't :/ no request with 'analytics.js' in the URL.
>
> I think the reason is that tracking protection blocks Google Analytics.
> There is no reason to expect analytics.js to show up in the log. ;-)
Quite the opposite! We always create channels even for trackers when TP is on. The annotation/blocking happens inside the channel during the opening process. The channel is instantiated and assigned a URL. Hence, if you load the log to [1], you MUST find nsHttpChannel with url containing "analytics.js", even though it's been later blocked. If you don't it means there has not been a channel created for that URL. Hence my question about the script loader and suspicion that something is blocking the request (the channel) before its creation.
[1] https://janbambas.cz/moz/logan/
(In reply to :Ehsan Akhgari from comment #18)
> Here is some more information. The "async-hide" class comes from Google
> Optimize: <https://developers.google.com/optimize/>.
>
> See this code snippet:
Yes, that's what I've found in bug 1420885 too.
(In reply to :Ehsan Akhgari from comment #19)
> I thought of one change in behavior that bug 1478539 has caused here which I
> filed as bug 1485182. I'm fairly certain that this bug is a duplicate of
> that bug in fact. Here is why.
>
> If you test this with Firefox release in a new profile, both spiegel.de and
> kino.de demonstrate the visible load delay, which is I think caused by the
> code in comment 18.
>
> However, previously, if you had uBlock Origin installed, for some reason
> this delayed behavior wouldn't happen. I _think_ the reason is that uBlock
> Origin actually doesn't block these scripts, and provides a shim replacement
> instead.
The dynamically loaded "experiment processing" (or whatever) script has to remove the class from <body>. Maybe the shim is doing that, but one would have to look.
> However when bug 1478539 landed, tracking protection started to
> cancel the channels before uBlock Origin had a chance to examine them, and
> as a result the same behavior that was visible without uBlock Origin started
> to appear.
Sounds reasonable!
>
> With the patch on bug 1485182 applied, on a new profile *without* uBlock
> Origin, the visible delay is present, but *with* uBlock Origin, there is no
> visible delay.
>
> I decided to file a separate bug to give Jan a chance to test this out in
> case there is more than one behavior change hiding here...
|
||
Comment 21•7 years ago
|
||
> I _think_ the reason is that uBlock Origin actually doesn't block these
> scripts, and provides a shim replacement instead.
Just to confirm: yes, this is the case, for Google Analytics/Google Tag Manager scripts, uBO redirects the network requests to local, neutered versions. These local versions test for the presence of window.dataLayer.hide object and if present, calls the window.dataLayer.hide.end() method[1], which gets rid of the artificial delay.
***
[1] https://github.com/uBlockOrigin/uAssets/blob/be747839a7b4399c04102f5f49b995afdd9b334e/filters/resources.txt#L583-L585
|
|
Assignee | |
Comment 22•7 years ago
|
||
(In reply to rhill@raymondhill.net from comment #21)
> > I _think_ the reason is that uBlock Origin actually doesn't block these
> > scripts, and provides a shim replacement instead.
>
> Just to confirm: yes, this is the case, for Google Analytics/Google Tag
> Manager scripts, uBO redirects the network requests to local, neutered
> versions. These local versions test for the presence of
> window.dataLayer.hide object and if present, calls the
> window.dataLayer.hide.end() method[1], which gets rid of the artificial
> delay.
>
> ***
>
> [1]
> https://github.com/uBlockOrigin/uAssets/blob/
> be747839a7b4399c04102f5f49b995afdd9b334e/filters/resources.txt#L583-L585
Makes perfect sense, thanks!
|
|
Assignee | |
Comment 23•7 years ago
|
||
Jan, I've landed the fixes for bug 1485182 on inbound, and they should merge into the next Nightly I hope very soon. I would appreciate if you can retest when that happens and let me know if the behavior prior to landing of bug 1478539 has been restored for you or not.
Thank you!
Flags: needinfo?(jan)
|
|
Reporter | |
Comment 24•7 years ago
|
||
(In reply to :Ehsan Akhgari from comment #23)
> Jan, I've landed the fixes for bug 1485182 on inbound, and they should merge into the next Nightly I hope very soon. I would appreciate if you can retest when that happens and let me know if the behavior prior to landing of bug 1478539 has been restored for you or not.
Verified fixed in Nightly 63 x64 20180822221004 de_DE @ Debian Testing. Thank you!
Status: NEW → RESOLVED
Closed: 7 years ago
Flags: needinfo?(jan)
OS: Linux → All
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
|
|
Assignee | |
Comment 25•7 years ago
|
||
Thanks a lot for taking the time to file the bug, and for testing this again! Now, we have an automated test for this, so that we won't cause this regression ever again. :-)
|
|
||
Updated•7 years ago
|
Flags: qe-verify+
|
||
Comment 26•7 years ago
|
||
I successfully reproduced the issue on Nightly 63.0a1 (2018-08-20) under Windows 10 (x64) using STR from Comment 0.
The issue is verified fixed on Firefox Beta 63.0b9. Tests were performed under Windows 10 (x64), Ubuntu 16.04 (x64) and macOS 10.12.
You need to log in
before you can comment on or make changes to this bug.
Description
•