1516552 - ETP Strict delays the visibility of all site content because of blocked GA/GTM resources

Status: RESOLVED FIXED

(Core :: Privacy: Anti-Tracking, defect, P3)

Description

[Vicky Chin [:vchin]]

STR:
1. Nagivate to www.theverge.com

Profile:
https://perfht.ml/2SqkKB2

On MacOSX Nightly 66.0a1 (2018-12-23) (64-bit) it takes a long time 4.5s for the first contentful paint to happen. Compared to Chrome which only takes 1 second. The profile shows the snippet when enter is hit to navigate to the page to when contentful paint happens. It seems we're idle for a good chunk of that time?

Comment 1

[Vicky Chin [:vchin]]

Opening against layout for now. dholbert, thoughts?

Comment 2

[Daniel Holbert [:dholbert]]

I don't see layout or styling taking any significant amount of time before that first paint, so this probably belongs in a different component.

A few observations:
 - the compositor process seems pretty continuously busy from ~3.5s to ~7s, even though nothing's been painted to the screen.
 - the content process is generally pretty quiet, but has a long paint at around 8s (a rasterize of 150ms in duration), and three kinda-long paints (65-77ms) earlier on.
 - the parent process seems to be spending a lot of time with "paint" in the backtrace, too.

This probably belongs in Graphics (or Graphics::Layers) since painting is what I'm seeing as the time-consuming thing here, and the compositor process churn seems maybe-worrisome.

--> Punting to Bas and mstange (who's away for a few weeks, hence also Bas in case he can get to it sooner.)

Comment 3

[Bas Schouten (:bas.schouten)]

(In reply to Daniel Holbert [:dholbert] from comment #2)
> I don't see layout or styling taking any significant amount of time before
> that first paint, so this probably belongs in a different component.
> 
> A few observations:
>  - the compositor process seems pretty continuously busy from ~3.5s to ~7s,
> even though nothing's been painted to the screen.
>  - the content process is generally pretty quiet, but has a long paint at
> around 8s (a rasterize of 150ms in duration), and three kinda-long paints
> (65-77ms) earlier on.
>  - the parent process seems to be spending a lot of time with "paint" in the
> backtrace, too.
> 
> This probably belongs in Graphics (or Graphics::Layers) since painting is
> what I'm seeing as the time-consuming thing here, and the compositor process
> churn seems maybe-worrisome.
> 
> --> Punting to Bas and mstange (who's away for a few weeks, hence also Bas
> in case he can get to it sooner.)

The compositor thread (there's no GPU process since this is OS X) is just busy re-uploading stuff here that's sent there in rapid transactions, this is also the work seen in the parent thread. We know uploading to be slow on OS X and it certainly isn't great, but that's -not- the reason for the long wait here.

If you look at the screenshots there is also -nothing showing- for all these paints that we're doing.

The long rasterizes, again, aren't great, but not wholly unexpected, if you look at what it's doing, it's basically spending 96% of its time clearing surfaces for newly created layers, perhaps the layerization here is suboptimal, which again, would be a layout issue. There certainly isn't anything left to do here for Gfx, it's basically a memset. I actually assume this is mostly page-faulting.

What's really concerning here though is that we spend several seconds just displaying black, no new obvious content comes in over the network, nor is any real work done in JS, and then all of a sudden a paint is generated that actually -does- draw the page, which seems to point at some sort of scheduling or invalidation issue. Again pointing towards something layout, or DOM-ish.

In short, yes, graphics takes some time here, but that time doesn't appear to be graphics' fault, nor is it the biggest concern here. It's the time we're -not- doing anything that's concerning.

Comment 4

[Daniel Holbert [:dholbert]]

Agreed that that the wait is the concerning bit. Not sure what we can do to investigate that -- mstange, do you know?

Personally I'm on Linux & I can't really reproduce this -- theverge.com starts painting content in ~1 second or less in Firefox Nightly on my linux box.  (Chrome feels slightly faster, but not drastically so.)

Bas, can you reproduce? (IIRC you're on Windows -- does it happen there?)

Vicky, does this happen every time for you? (even in a fresh profile with nothing much else running & no other tabs open? [just to rule out sources of background-CPU-usage during the "dark time" in the profile)

Comment 5

[Bas Schouten (:bas.schouten)]

(In reply to Daniel Holbert [:dholbert] from comment #4)
> Agreed that that the wait is the concerning bit. Not sure what we can do to
> investigate that -- mstange, do you know?
> 
> Personally I'm on Linux & I can't really reproduce this -- theverge.com
> starts painting content in ~1 second or less in Firefox Nightly on my linux
> box.  (Chrome feels slightly faster, but not drastically so.)
> 
> Bas, can you reproduce? (IIRC you're on Windows -- does it happen there?)
> 
> Vicky, does this happen every time for you? (even in a fresh profile with
> nothing much else running & no other tabs open? [just to rule out sources of
> background-CPU-usage during the "dark time" in the profile)

I don't have screenshots on Windows sadly so it's hard for me to tell how bad it is here, but indeed it doesn't seem to take more than 2 seconds. What I can tell as a difference between Firefox and Chrome, is that Firefox definitely draws 'all black' first, and then the actual content in some subsequent frame. Chrome draws all the content right away.

Chrome also -does- draw the page considerably faster on my machine (even though we're nowhere near as slow for me as described on the bug).

I'd say considering the compositor and content process main thread happily being scheduled here, as well as no gaps in profiler samples. Background CPU usage is highly unlikely. It could be, although unlikely, we're waiting for some other thread in Firefox I suppose.

Comment 6

[Vicky Chin [:vchin]]

I still get the 2 seconds of black screen on a new profile. It loads slightly faster on a new profile but it could be because I had the profiler running.

Comment 7

[Daniel Holbert [:dholbert]]

(In reply to Bas Schouten (:bas.schouten) from comment #5)
> What I can tell as a difference between Firefox and Chrome, is that Firefox
> definitely draws 'all black' first, and then the actual content in some
> subsequent frame. Chrome draws all the content right away.

On my linux machine, Chrome draws "all black" first, just like Firefox does, FWIW.

Comment 8

[Daniel Holbert [:dholbert]]

OK, so I think content blocking may be involved here.

Specifically, starting with a fresh profile...
 (1) If I visit about:preferences#privacy and change Content Blocking to "strict", then I get a nearly 4-second blank black page before the first contentful paint.

 (2) If I reset Content Blocking to "Standard" or disable it entirely, then I see a ~0.5 second blank black page before the first contentful paint.

 (3) If I install uBlock origin, then the first contentful paint happens "instantly" - I don't see any black blank page at all.

Comment 9

[Daniel Holbert [:dholbert]]

Here's a profile of scenario (1), with strict content blocking enabled: https://perfht.ml/2VeYFHC
The page starts loading at 1 second (and paints a blank black background pretty quickly), and the content doesn't paint until ~5.5 seconds.

If I look at the DOM during the "blank page" time, I can see that they have:
 <html class="async-hide fonts-loaded">
...where .async-hide is an inline style rule which applies "opacity: 0 !important"

I also see this script in their HTML:
(function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;h.start=1*new Date;
h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')};
(a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;
})(window,document.documentElement,'async-hide','dataLayer',4000,
{'GTM-54FC4VZ':true});

It's somewhat obfuscated, but given the observed behavior, I'm guessing that JS adds this class (hiding all the content), and then waits 4 seconds before removing it. They probably have some other scripts that remove it sooner in the usual case, but they fallback to a setTimeout(...,4000) to be sure their content eventually shows up.  And perhaps Firefox's strict content blocking causes that "usual case" to break, by removing some link in their expected chain that leads up to removing the async-hide class.

So that explains why the page is dark for 4 seconds, probably; there's nothing to paint.  So the question is why Vicky is hitting that scenario.  Vicky, do you get better results (less than 4 seconds) if you disable content blocking? (by clicking the left edge of the URL bar and choosing "Turn off blocking for this site")

Comment 10

[[e-mail]]

This sounds like the artificial delay introduced by Google APIs and blocking `googletagmanager.com` causes the delay to take longer.

I can reproduce the delay if blocking `googletagmanager.com` with uMatrix while allowing first-party scripts to execute. The delay is a commonly reported issue when blocking `googletagmanager.com`, see https://github.com/uBlockOrigin/uMatrix-issues/issues/93#issuecomment-449982859

Comment 11

[Daniel Holbert [:dholbert]]

Thanks!  I do see references to googletagmanager.com in a saved snapshot of the page. And my observations (class name & timeout duration) in comment 9 seem to jive with this comment on the uBlockOrigin github issue:
> Google's optimisation was async-hide's opacity set with a delay of 4 seconds, wasn't it ?
https://github.com/uBlockOrigin/uMatrix-issues/issues/93#issuecomment-449980342

So, perhaps we should investigate if our "standard" content-blocking configuration (on Vicky's machine at least) is blocking that domain for some reason, & if we can suppress that from happening, or something. [EDIT: Turns out Vicky was only seeing the issue with "custom" blocking that was similar to "strict", per comment 13.]

Comment 12

[Bas Schouten (:bas.schouten)]

This issue was based on an issue originally reported by Ekr. This suggests that at least for this site it's relatively widespread. I had no idea this class of issues existed but it certainly seems worrying to me, especially since as far as I can tell we have no telemetry or any other data from out in the world to tell us how prevalent this is, and it will cause a significant difference in perceived page load performance compared to Chrome.

NI'ing bz, not because I expect him to act on this but I want to make sure we're aware of this 'class' of performance issue.

Comment 13

[Vicky Chin [:vchin]]

:dholbert is right. I thought I changed to standard content blocking but I was on custom blocking that was equivalent to strict blocking. The "blocking trackers in all windows" is what causes the long black screen paint. The content blocking list used in all windows is Disconnect.me basic protection. If i change that to only be in private windows then I only see a split second of black before the rest of the page paints.

Comment 14

[Daniel Holbert [:dholbert]]

Thanks, Vicky. It sounds like this isn't a problem in our default configuration, then -- only users that have opted in to stricter content-blocking.

It's still worth investigating what we can do here, though there's potentially a small number of (privacy/perf-conscious) users who may be affected - not our whole userbase.

Comment 15

[(no longer active)]

(In reply to Daniel Holbert [:dholbert] from comment #9)
> Here's a profile of scenario (1), with strict content blocking enabled:
> https://perfht.ml/2VeYFHC
> The page starts loading at 1 second (and paints a blank black background
> pretty quickly), and the content doesn't paint until ~5.5 seconds.
> 
> If I look at the DOM during the "blank page" time, I can see that they have:
>  <html class="async-hide fonts-loaded">
> ...where .async-hide is an inline style rule which applies "opacity: 0
> !important"
> 
> I also see this script in their HTML:
> (function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;h.start=1*new Date;
> h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')};
> (a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;
> })(window,document.documentElement,'async-hide','dataLayer',4000,
> {'GTM-54FC4VZ':true});
> 
> It's somewhat obfuscated, but given the observed behavior, I'm guessing that
> JS adds this class (hiding all the content), and then waits 4 seconds before
> removing it. They probably have some other scripts that remove it sooner in
> the usual case, but they fallback to a setTimeout(...,4000) to be sure their
> content eventually shows up.  And perhaps Firefox's strict content blocking
> causes that "usual case" to break, by removing some link in their expected
> chain that leads up to removing the async-hide class.
> 
> So that explains why the page is dark for 4 seconds, probably; there's
> nothing to paint.  So the question is why Vicky is hitting that scenario. 
> Vicky, do you get better results (less than 4 seconds) if you disable
> content blocking? (by clicking the left edge of the URL bar and choosing
> "Turn off blocking for this site")

This is a known issue, it's caused by the interaction of Google Optimize and Google Analytics/Tag Manager (which we block using tracking protection), see https://bugzilla.mozilla.org/show_bug.cgi?id=1484713#c18 (note that bug 1484713 itself is about that issue, but in that issue I described what's going on.  There are other reports of similar issues on other sites on Bugzilla as well.

The solution to this bug would be to provide a surrogate replacement for the GA/GTM scripts, similar to privacy extensions like uBlock Origin and Privacy Badger.  We've talked about this in the anti-tracking project but looking at Bugzilla I see that there is no bug on file for it yet.  Should I repurpose this bug?  (If not, we should file another one and make it depend on it, although this is going to be a dupe of that one effectively.)

(In reply to Daniel Holbert [:dholbert] from comment #14)
> Thanks, Vicky. It sounds like this isn't a problem in our default
> configuration, then -- only users that have opted in to stricter
> content-blocking.

That's not exactly right, this is very much the default configuration of Firefox in private browsing, and has been so since Firefox 42, so it is a problem that users are facing right now.

Comment 16

[(no longer active)]

BTW in that bug you can find two other sites affected: spiegel.de and kino.de.  Any site using GA/GTM + Google Optimize will be similarly affected right now in Private Browsing mode.

Comment 17

[Vicky Chin [:vchin]]

(In reply to :Ehsan Akhgari from comment #15)

> 
> The solution to this bug would be to provide a surrogate replacement for the
> GA/GTM scripts, similar to privacy extensions like uBlock Origin and Privacy
> Badger.  We've talked about this in the anti-tracking project but looking at
> Bugzilla I see that there is no bug on file for it yet.  Should I repurpose
> this bug?  (If not, we should file another one and make it depend on it,
> although this is going to be a dupe of that one effectively.)

I think it's ok to repurpose this bug.

Comment 18

[Daniel Holbert [:dholbert]]

(In reply to Bas Schouten (:bas.schouten) from comment #12)
> NI'ing bz, not because I expect him to act on this but I want to make sure
> we're aware of this 'class' of performance issue.

(clearing ni to lighten bz's backlog, since per comment 15, it seems we have a decent awareness & understanding of this class of issue)

Comment 20

[Boris Zbarsky [:bzbarsky]]

Note that at least for me on Mac, we do end up with a black paint, then a paint with content,  even with tracking protection off, whereas Chrome just paints with content.  Is there a separate tracker for that?

Comment 21

[(no longer active)]

(In reply to Boris Zbarsky [:bzbarsky, bz on IRC] from comment #20)

Note that at least for me on Mac, we do end up with a black paint, then a
paint with content, even with tracking protection off, whereas Chrome just
paints with content. Is there a separate tracker for that?

Not that I know of, please file one!

Comment 24

[Boris Zbarsky [:bzbarsky]]

Not that I know of, please file one!

Filed bug 1533102

Comment 27

[Thomas Boyt]

As a web developer who's recently implemented Google Optimize and the anti-flicker script on a website, and a relatively new Firefox user, I think implementing a no-op version of the Google Analytics script to immediately call dataLayer.hide.end() and remove the async-hide class, as uBlock does, is a very important web compat feature.

When I first started using Firefox with content blocking, I'd noticed that many sites were taking a mysteriously long time to render, and assumed it had something to do with the blocking, but never figured out why. The moment I implemented Optimize's anti-flickering snippet and opened my updated site in Firefox for the first time, though, I instantly realized what I'd been seeing for several months. The free version of Google Optimize is a relatively new product and one of (if not the) only free client-side options developers have for A/B testing, and I imagine usage of it will only increase over time, with more sites feeling "mysteriously slow to render" in Firefox.

As a user, of course, I can install uBlock for now. As a developer, though, I'm very concerned: if I understand the new default Enhanced Tracking Protection settings correctly, Google Analytics is now blocked by default, which makes this an issue for every new Firefox user (not just private browsers or user with custom settings, as was previously the case). Right now, I have to worry about Firefox users seeing delayed renders of the site.

For now, I've simply lowered the script's timeout from 4 to 2 seconds to make the site slightly more usable in Firefox. I'm also looking into whether Google Optimize can be used without Google Analytics, so it can still work with Enhanced Tracking Protection enabled. Otherwise, if the default settings for new Firefox users become the default settings for all Firefox users, I may just try to update the script to do a User Agent sniff to disable the flicker script in Firefox (since presumedly Optimize wouldn't be able to work anyways).

Comment 28

[(no longer active)]

(In reply to Thomas Boyt from comment #27)

As a web developer who's recently implemented Google Optimize and the anti-flicker script on a website, and a relatively new Firefox user, I think implementing a no-op version of the Google Analytics script to immediately call dataLayer.hide.end() and remove the async-hide class, as uBlock does, is a very important web compat feature.

When I first started using Firefox with content blocking, I'd noticed that many sites were taking a mysteriously long time to render, and assumed it had something to do with the blocking, but never figured out why. The moment I implemented Optimize's anti-flickering snippet and opened my updated site in Firefox for the first time, though, I instantly realized what I'd been seeing for several months. The free version of Google Optimize is a relatively new product and one of (if not the) only free client-side options developers have for A/B testing, and I imagine usage of it will only increase over time, with more sites feeling "mysteriously slow to render" in Firefox.

Yes, definitely agreed on the importance of fixing this issue. It is just a matter of prioritization given the sheer number of things we work on from a web compat perspective. FWIW I am really hoping to find some time to fix this in the next couple of months, it's very high on the list of things I'd like to fix among the breakage caused by our Tracking Protection. That being said, it is sort of an on-the-side project for me right now. :-/

As a user, of course, I can install uBlock for now. As a developer, though, I'm very concerned: if I understand the new default Enhanced Tracking Protection settings correctly, Google Analytics is now blocked by default, which makes this an issue for every new Firefox user (not just private browsers or user with custom settings, as was previously the case). Right now, I have to worry about Firefox users seeing delayed renders of the site.

No, that's not correct. Enhanced Tracking Protection is a completely different feature than Tracking Protection (sorry about the name confusion!). Tracking Protection is a much older feature in Firefox, which works by blocking the resources coming from domains on the Disconnect TP list from loading, which causes a lot of breakage, and this bug is one example of that. Since 2015, Tracking Protection has been turned on by default in Private Browsing for Firefox users. Since Firefox 57 it was offered as a setting that users could turn on themselves. Since a few versions ago (maybe 63 or 64?) it has also been exposed as a non-default setting under Content Blocking called Strict.

Enhanced Tracking Protection is a feature that works by blocking access to cookies, storage and communication APIs from third-party trackers on the Disconnect TP list which we've been working on since last year and is now the default for new users and will become the default for existing users soon. Enhanced Tracking Protection has very few known breakages. Specifically, since it doesn't block any scripts from loading, it doesn't cause the current issue at hand.

For now, I've simply lowered the script's timeout from 4 to 2 seconds to make the site slightly more usable in Firefox. I'm also looking into whether Google Optimize can be used without Google Analytics, so it can still work with Enhanced Tracking Protection enabled. Otherwise, if the default settings for new Firefox users become the default settings for all Firefox users, I may just try to update the script to do a User Agent sniff to disable the flicker script in Firefox (since presumedly Optimize wouldn't be able to work anyways).

I hope the above explanation helps make things more clear. ETP shouldn't have any impact around this issue when it becomes the default for all users, so I don't think the User Agent sniffing workaround that you mentioned is necessary at all. By the way, in order to actually test this to be sure, you can always use Firefox Nightly or Developer Edition which both have ETP on by default to double check and make sure your site works in them as expected. (Of course, this bug would still reproduce in private windows, as I mentioned above, since Tracking Protection is turned on alongside with Enhanced Tracking Protection for those windows!)

Comment 29

[Thomas Boyt]

(In reply to :Ehsan Akhgari from comment #28)

Ah, thanks for the clarifications! Sorry for the false alarm; I didn't realize my Firefox content blocking settings didn't map to the new defaults. I didn't understand exactly how the standard/strict/custom settings mapped to what gets enabled and disabled, so I appreciate the insight.

I've tested my site in Firefox Dev Edition with default settings and GA loads fine (other than in private mode, of course).

Comment 32

[Randell Jesup [:jesup] (needinfo me)]

ehsan - is this still something you want to work on? If so, what priority is it? (Can you set the priority on this bug?) We have it marked as a p1:pageload for the perf team currently; should we keep that, or since this is for TP (and not ETP), should we make it a p2:pageload?
Thanks

Comment 33

[(no longer active)]

(In reply to Randell Jesup [:jesup] (needinfo me) from comment #32)

ehsan - is this still something you want to work on?

want to, sure. :-) I've unfortunately been unable to find the time to do that.

If so, what priority is it? (Can you set the priority on this bug?)

The unfortunate reality of the moment is that this bug falls in the giant basket of breakage issues caused by strict content blocking, which are all unowned, which means that it'll be prioritized as P3. This is from the perspective of the privacy team, which means it won't necessarily be worked on in the short term future.

(Separately we've been talking about changing this situation but that is where we are right now.)

We have it marked as a p1:pageload for the perf team currently; should we keep that, or since this is for TP (and not ETP), should we make it a p2:pageload?

The priority of this from a performance perspective isn't something that I'm comfortable commenting on... We've been coming across duplicates of this on and off-Bugzilla regularly so I can say that it is something a good number of users hit in practice, but whether that means p1:pageload or p2:pageload I suppose depends on how these flags are used on other similar bugs, which is something I'm not too informed on these days, so I'll refrain from saying anything about that.

Comment 34

[Olivier Vit (just a reporter)]

Hi,

GTM is a sort of CMS for tags and javascript code, why would block it ? Do you block wordpress ? It doesn't set cookies nor tracks anything by itself.

See https://github.com/disconnectme/disconnect-tracking-protection/commit/ac72379301ae4528f83e2110546b18e1d08d80d2

If tags within GTM need to be blocked, they'll be on your tag list.

But when site owners decide to use GTM to deploy code, as well as consent management modules, blocking GTM is hardly hurting usability and site's featureset as well as the ability to respect user choices in terms of privacy.

It's just a non sense to block GTM by itself

Comment 35

[Daniel Holbert [:dholbert]]

(In reply to Olivier Vit (just a reporter) from comment #34)

See https://github.com/disconnectme/disconnect-tracking-protection/commit/ac72379301ae4528f83e2110546b18e1d08d80d2

Note that that commit was after this bug was filed/investigated. (Comments 10-15 were from Dec 2018 - Jan 2019; whereas the the disconnect.me commit that you referenced here was in April 2019)

I believe we largely defer to disconnect.me on classifying tracking vs. non-tracking, so if they've changed their categorization of GTM (googletagmanager.com), then presumably we have as well.

And it looks to me like we are allowing googletagmanager.com, though the "shield" UI shows me that we do block googletagservices.com (note "services", not "manager"), presumably because disconnect does still have that one flagged.

And I am still seeing this delay happening (with "async-hide" on the body until it gets removed after an arbitrary/annoying delay). Maybe googletagservices is really (or now) responsible for doing that?

Here's a profile of this happening on theverge.com in today's nightly with strict tracking protection: https://perfht.ml/2TqbjV5

Comment 36

[Daniel Holbert [:dholbert]]

(Looking at theverge.com source, maybe this "async-hide" junk has now moved to "Google Analytics" and/or "Google Optimize" (if that's a thing)?

At least, here's the relevant bit of theverge.com's raw html:

<!-- Google Optimize -->
<!-- Page hiding snippet -->
<style>.async-hide { opacity: 0 !important} </style>
<script>(function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;h.start=1*new Date;
h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')};
(a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;
})(window,document.documentElement,'async-hide','dataLayer',4000,
{'GTM-54FC4VZ':true});</script>

<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
  ga('create', 'UA-26533115-1', 'auto', {'allowLinker': true,'useAmpClientId': true});
  ga('require', 'GTM-54FC4VZ');
</script>
<!-- End Google Optimize -->

Comment 37

[Daniel Holbert [:dholbert]]

It looks like this "page hiding snippet" is intended as an anti-flicker technique, and is documented at https://developers.google.com/optimize/ and it indeed depends on Google Analytics (and falls back to 4-second timeout when that's not present). And google analytics is currently blocked by disconnect.me:
https://github.com/disconnectme/disconnect-tracking-protection/blame/bfd4a01d791ccdf1d34d4a8ec64a678d01f2fd24/entities.json#L5367

So googletagmanager.com is no longer relevant here -- now this artificial delay is coming from a dependency on Google Analytics (at least in this case at theverge.com).

Comment 38

[Dmitry]

(In reply to Daniel Holbert [:dholbert] from comment #37)

It looks like this "page hiding snippet" is intended as an anti-flicker technique, and is documented at https://developers.google.com/optimize/ and it indeed depends on Google Analytics (and falls back to 4-second timeout when that's not present). And google analytics is currently blocked

Hi Daniel,

I may be wrong, but I don't thinks that's the case. As per Google Optimize's Install the anti-flicker snippet reference:

If you're using Google Tag Manager to deploy Optimize, replace CONTAINER_ID with your Tag Manager container ID.

So {'GTM-54FC4VZ':true} in the <!-- Page hiding snippet --> above has nothing to do with Google Analytics:
The snippet hides the page until Google Tag Manager is loaded.

As far as I understand it, there shouldn't be any Page Load delay in Firefox, since Firefox blocks Google Analytics, not Google Tag Manager.
Yes, once Google Tag Manager has loaded, it may fire Google Analytics which may/may not be blocked. But that's another story.

Comment 39

[Daniel Holbert [:dholbert]]

(In reply to Dmitry from comment #38)

So {'GTM-54FC4VZ':true} in the <!-- Page hiding snippet --> above has nothing to do with Google Analytics:
The snippet hides the page until Google Tag Manager is loaded.

The piece of the snippet in comment 36 that I'd was suggesting was "dependent on Google Analytics" was this piece:

(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
[...]
  ga('require', 'GTM-54FC4VZ');

That seems to be loading Google Analytics (loads it in the first line, and then passes this GTM-54FC4VZ GTM token to it later on). The whole snippet is kinda obfuscated/minified, so I don't know exactly what it's doing, but there's clearly an association there.

(In reply to Dmitry from comment #38)

As far as I understand it, there shouldn't be any Page Load delay in Firefox, since Firefox blocks Google Analytics, not Google Tag Manager.

And yet, there is still a page load delay. :) (for me at least)

I don't know for sure which resource-block is causing the issue here, but some resource block is preventing the page from loading (or rather, there's some resource that the page is assuming will remove the async-hide class, and that resource is blocked or failing to do its thing, which means we fall back on the page's own 4000ms timeout graceful-fallback for removing that class). And the GTM script is not being blocked, so it's not that.

Comment 40

[(no longer active)]

As comment 15 indicated a long time ago, this bug was always about the interaction of the blocking of Google Optimize with either Google Analytics/Google Tag Manager. GTM is no longer blocked by Disconnect, but GA still is, so sites which use it in addition to Google Optimize will experience this problem in Firefox.

I'm going to go ahead and mark several comments which are talking about tangential issues unrelated to the main problem being discussed here and/or repeating already discussed information as obsolete to make the bug easier to follow. Since this bug is getting harder and harder to understand for the engineers following it, unless you have something to add which directly helps in fixing this bug, I'd appreciate if you would kindly refrain from adding more comments, as that is unhelpful in driving this closer to a resolution. Thanks!

Comment 42

[Steven Englehardt [:englehardt]]

This bug is simultaneously serving as a tracking bug for GA/GTM blocking issues and all of our implementation work for surrogate/shim script replacement. I've found this pretty confusing, and I suspect anyone new to the project would also find this confusing. I've decided to repurpose this bug back to focusing on the GTM/GA first paint breakage and have filed Bug 1648863 to track our overall shim script implementation work (which will address many types of breakage, not just GTM/GA).

Comment 44

[Markus Stange [:mstange]]

I'm going to look into this. We have the ability to shim resources now, and we even already have a relevant shim at https://searchfox.org/mozilla-central/source/browser/extensions/webcompat/shims/google-analytics-tag-manager.js , but for some reason it's not working properly.

Comment 45

[Markus Stange [:mstange]]

The problem seems to be that both https://searchfox.org/mozilla-central/source/browser/extensions/webcompat/shims/google-analytics-tag-manager.js and https://searchfox.org/mozilla-central/source/browser/extensions/webcompat/shims/google-analytics.js only call window.dataLayer.hide.end() if window.ga is not defined. But on the pages that I've tested (theverge.com and t-mobile.cz), window.ga is already defined by another inline script in the main html page.
The uBlock Origin shim calls window.dataLayer.hide.end() even if window.ga is already defined.

Comment 46

[Markus Stange [:mstange]]

Attachment: Bug 1516552 - In the Google Analytics shim, call window.dataLayer.hide.end() even if window.ga is already defined. r=twisniewski

This also removes the queue loop at the end, because as far as I can tell that code was never run.
The loop was only entered if gaQueue was falsy (check in the first line), and if gaQueue was a
function (check just before the loop), which is unlikely to be true at the same time.

The uBlock Origin loop does have that loop, but it doesn't call the original function; the loop
calls the shim function, because uBlock Origin replaces window.ga with its shim function even
if window.ga is already defined.

Comment 47

[Markus Stange [:mstange]]

I have checked all the duplicates of this bug and the patch seems to fix all of them.

Comment 48

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b2473d585fff
In the Google Analytics shim, call window.dataLayer.hide.end() even if window.ga is already defined. r=twisniewski,webcompat-reviewers

Comment 49

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/b2473d585fff