Closed Bug 2008101 Opened 6 months ago Closed 6 months ago

privacy.annotate_channels.strict_list.enabled is enabled on all Fenix channels, leads to slower page load

Categories

(Core :: Privacy: Anti-Tracking, defect, P2)

Product:
Core
Component:
Privacy: Anti-Tracking
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
148 Branch
Tracking Flags:
Tracking Status
firefox147 --- fixed
firefox148 --- fixed

People

(Reporter: acreskey, Assigned: bvandersloot)

References

Details

Attachments

(4 files)

Bug 2008101 - Disable privacy.annotate_channels.strict_list.enabled on ETP standard Fenix channels - r=timhuang!
48 bytes, text/x-phabricator-request
Details | Review
comparison_target.mp4
286.50 KB, video/mp4
Details
comparison_spotify.mp4
651.76 KB, video/mp4
Details
Bug 2008101 - Disable privacy.annotate_channels.strict_list.enabled on ETP standard Fenix channels -
48 bytes, text/x-phabricator-request
phab-bot
: approval-mozilla-release+
Details | Review

In bug 2006169 we discovered that privacy.annotate_channels.strict_list.enabled is set to true on all channels of Fenix downloaded from the Play Store.
Confirmed in Nightly and Beta via about:config and in Fenix Release via remote debugging, Multiprocess Toolbox:

Services.prefs.getBoolPref("privacy.annotate_channels.strict_list.enabled")
true

As a consequence, requests for many common web resources are annotated as trackers and thus tailed, a process by which we defer their loading.

Example resources include google fonts, JS and CSS frameworks hosted on Cloudflare, embedded YouTube players, etc.
More are listed in
https://bugzilla.mozilla.org/show_bug.cgi?id=1915490#c0
and
https://bugzilla.mozilla.org/show_bug.cgi?id=2006169#c5

Tailing these resources has a negative impact on pageload performance

I propose we fix this asap and consider uplifting the change.

Flags: needinfo?(tihuang)

Benjamin and I are discussing this in slack.

Copied/summarized notes: (note that I'm not a domain expert here)

Something ... happens here: https://searchfox.org/firefox-main/source/mobile/android/fenix/app/src/main/java/org/mozilla/fenix/gecko/GeckoProvider.kt#94

I'm really not familiar enough with ETP and our Android components/Fenix configuration. but it looks like there may be a bug here:
https://searchfox.org/firefox-main/rev/fb84c9bccbfc34b91613ff9f3c955a887c7e7196/mobile/android/android-components/components/browser/engine-gecko/src/main/java/mozilla/components/browser/engine/gecko/ext/TrackingProtectionPolicy.kt#64-72

This returns EtpLevel.STRICT for all cases except TrackingCategory.NONE.

And this propagates to mEtpStrict
https://searchfox.org/firefox-main/rev/fb84c9bccbfc34b91613ff9f3c955a887c7e7196/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/ContentBlocking.java#785

And the .commit() makes the Pref<Boolean> Pref<Boolean> mEtpStrict overwrite the gecko pref, privacy.annotate_channels.strict_list.enabled
https://searchfox.org/firefox-main/rev/fb84c9bccbfc34b91613ff9f3c955a887c7e7196/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/ContentBlocking.java#464-465

Assignee: nobody → bvandersloot
Status: NEW → ASSIGNED

Throwing up a patch as a candidate that may fix things.

Assignee: bvandersloot → nobody
Status: ASSIGNED → NEW
Assignee: nobody → bvandersloot
Status: NEW → ASSIGNED

I'm not sure if our raptor-browsertime pageload tests will pickup differences from this change, but I've pushed a performance comparison with Ben's patch here:
https://perf.compare/compare-lando-results?baseLando=170947&newLando=170948&baseRepo=try&newRepo=try&framework=13
(Will take a day or so for results to arrive)
With this binary, I'll also re-evaluate the benchmark from bug 2006169

Looking at Fenix 100.1.0 Release, with a push date of April 25, 2022, from this archive,
https://archive.mozilla.org/pub/fenix/releases/

I'm seeing the same issue with privacy.annotate_channels.strict_list.enabled set to true.
Via about:debugging:

Services.prefs.getBoolPref("privacy.annotate_channels.strict_list.enabled")
true

So, as far as I can tell, this has been a longstanding issue.

Pushed by bvandersloot@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/45e1ba883f45 https://hg.mozilla.org/integration/autoland/rev/97883dfedafb Disable privacy.annotate_channels.strict_list.enabled on ETP standard Fenix channels - r=timhuang,android-reviewers,Roger
Pushed by csabou@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/91a51941ec69 https://hg.mozilla.org/integration/autoland/rev/979c982aea82 Revert "Bug 2008101 - Disable privacy.annotate_channels.strict_list.enabled on ETP standard Fenix channels - r=timhuang,android-reviewers,Roger" for causing android failures on setEnhancedTrackingProtectionLevel

Backed out for causing android failures on setEnhancedTrackingProtectionLevel

Push with failures

Failure log

Backout link

Flags: needinfo?(tihuang)

Confirming that the patch in https://phabricator.services.mozilla.com/D277702 is working as expected in that, once applied, privacy.annotate_channels.strict_list.enabled defaults to false on Fenix.

We can see that in the "Web Assets Benchmark", resources are no longer given the Tail class of service.
https://share.firefox.dev/44VXLoS

Performance wise, mean latency is improved by ~37%, median latency is improved by ~60%, and throughput is doubled.
(Note that most of the resources in this test happen to be classified as content trackers, so it's particularly sensitive to this change).

| Metric          | Baseline (3 runs)   | strict_list=false (3 runs)   | Change            |
|-----------------|---------------------|------------------------------|-------------------|
| Mean Latency    | 776.71 ms           | 492.02 ms                    | -36.7% (faster)   |
| Median Latency  | 614.17 ms           | 246.00 ms                    | -60.0% (faster)   |
| Mean Throughput | 3.44 Mbps           | 6.95 Mbps                    | +102% (2x faster) |

(In reply to Andrew Creskey [:acreskey] from comment #5)

I'm not sure if our raptor-browsertime pageload tests will pickup differences from this change, but I've pushed a performance comparison with Ben's patch here:
https://perf.compare/compare-lando-results?baseLando=170947&newLando=170948&baseRepo=try&newRepo=try&framework=13
(Will take a day or so for results to arrive)
With this binary, I'll also re-evaluate the benchmark from bug 2006169

As far as I can, the CI test will not be useful for evaluating the performance impact as Safebrowsing and TrackingProtection are disable by default in Browsertime:
https://github.com/sitespeedio/browsertime/blob/fd66684edf12d7e879ad6c6703e189219dde7f9f/lib/support/cli.js#L586-L597

See Also: → 2008355

As far as I can, the CI test will not be useful for evaluating the performance impact as Safebrowsing and TrackingProtection are disable by default in Browsertime:
https://github.com/sitespeedio/browsertime/blob/fd66684edf12d7e879ad6c6703e189219dde7f9f/lib/support/cli.js#L586-L597

Logged in bug 2008355

Pushed by bvandersloot@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/910fe151fd92 https://hg.mozilla.org/integration/autoland/rev/d7ab20bccecb Disable privacy.annotate_channels.strict_list.enabled on ETP standard Fenix channels - r=timhuang,android-reviewers,Roger

https://hg.mozilla.org/mozilla-central/rev/d7ab20bccecb

Status: ASSIGNED → RESOLVED
Closed: 6 months ago
status-firefox148: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 148 Branch

We may be able to observe a performance impact from this change via Fenix telemetry.

In the nightly channel I believe this will take a few more days as clients are slow to update to the latest build.
The fix has a pushdate of 21:20:01 on Jan 2, 2026 so I would expect that build dates of Jan 3, 2026 and onwards will include the change.

As of the latest telemetry only about a third of the pageload event submissions are from clients with the fix. query
And we will only be able to get up to about 75% within a week due to the long tail of adoption.

But if this change does lead to a measurable performance impact, we should eventually see it in pageload metrics by submission date.

However, I think a clearer way to assess impact is to plot the performance metrics by submission date, splitting the data into builds with the fix (Jan 3 and later) versus builds without the fix.
Query: page-load metrics by submission date, segmented pre-/post-fix.
This also helps control for potentially misleading day-of-week seasonality, since we’re comparing metrics within the same submission day.

With the caveat that these queries are new and so we may need to iterate on them.

If the fix has an effect, it should be also be more apparent in the higher-volume Beta and especially Release channels.

See Also: → 1652517
See Also: → 1618687

In terms of measuring performance impact of this fix:

  1. Telemetry
    Looking at the query in which we plot the performance metrics by submission date, splitting the data into builds before the fix and builds after the fix (jan 2, 2026) there is some indication of an improvement.
    However, there is a huge sampling bias here because we are also looking at the ~60% of Fenix nightly users who upgraded to the latest vs the 40% who didn't.
    So while promising, I think this is low-confidence data (although observing impact on Beta and Release will be helpful.)

  2. Automated performance suites
    Unfortunately I cannot do an automated sweep of top sites to measure impact because our performance test harness, browsertime, disables tracking protection and safeBrowsing features and so the issue is hidden in that environment. I just logged this in bug 2008355

  3. Manual examination of top mobile sites
    I performed a manual scan of select top-50 mobile sites and I did find some examples of significant improvements from this fix.
    On https://www.target.com/, what is effectively the hero asset is tailed and the request takes about 1.4 seconds to complete. profile
    Compare to a profile with the fix applied in which the request is made in ~340ms. profile.
    Spotify.com is another example as key assets are retrieved from google storage and thus tailed. Here the background image is delayed for over 2.5 seconds. Profile

Attached video comparison_target.mp4 —

This comparison shows the impact of tailing the hero image on https://www.target.com/
fenix_baseline, left, is Fenix behaviour prior to this fix.
fenix_strict_list_disabled, right, is our current behaviour, with this fix.

Timings will vary due to network conditions, etc, but this is a reproducible difference.

Attached video comparison_spotify.mp4 —

Comparison of https://newsroom.spotify.com/, before (left), and after (right) application of this fix.

(In reply to Andrew Creskey [:acreskey] from comment #16)

  1. Manual examination of top mobile sites
    I performed a manual scan of select top-50 mobile sites and I did find some examples of significant improvements from this fix.
    On https://www.target.com/, what is effectively the hero asset is tailed and the request takes about 1.4 seconds to complete. profile
    Compare to a profile with the fix applied in which the request is made in ~340ms. profile.
    Spotify.com is another example as key assets are retrieved from google storage and thus tailed. Here the background image is delayed for over 2.5 seconds. Profile

I should add that while it was common to see resources tailed on top sites (particularly google fonts or JS frameworks hosted on Cloudflare) in general these did not lead to noticeable performance issues (at least not that I could spot through manual checks).
But without automated tooling, it’s hard to quantify the real impact.

In any event, we're aligned that we want this fix, so I think we should deploy it to Fenix users in as quickly as is safe and prudent.

Benjamin, Tim — what are your thoughts on the potential risk of this change?
The Android performance folks suggested uplifting from Beta to Release so we can ship it in the Fenix 147 dot release on January 20.
https://whattrainisitnow.com/release/?version=147

Regardless, I think we should be preparing release notes, setting tracking flags, etc, in preparation.

Flags: needinfo?(tihuang)
Flags: needinfo?(bvandersloot)

Personally I think an uplift into the planned dot release is right. We'll have enough time in Beta to hopefully get a feel on guardrails, while still delivering before 148.

Flags: needinfo?(bvandersloot)

Benjamin, I'm curious, what's the privacy advantage of tailing these resources ever?

Flags: needinfo?(bvandersloot)

None. This is an old feature from before my time, but the idea is that tracking resources would not typically affect visible components. This assumption has been becoming less true lately, so it may be worth revisiting this feature.

Flags: needinfo?(bvandersloot)

Benjamin, do you know if we're also hitting this tailing code in private browsing mode?

Flags: needinfo?(bvandersloot)

Not with certainty. Tim may know off hand.

Flags: needinfo?(bvandersloot)

(In reply to Jeff Muizelaar [:jrmuizel] from comment #23)

Benjamin, do you know if we're also hitting this tailing code in private browsing mode?

Using the web-assets-benchmark from bug 2006169 I am seeing that we are hitting this tailing code in private browsing windows.

Captured on desktop:
https://share.firefox.dev/49dnOKN

Given the known major performance impact and no known privacy benefits, my suggestion is that we take this code out completely. Thoughts?

It seems reasonable to me, but I'd like Tim's take since he may have more historical context that I'm missing.

(In reply to Jeff Muizelaar [:jrmuizel] from comment #23)

Benjamin, do you know if we're also hitting this tailing code in private browsing mode?

Yes, we hit this tailing in PBM.

(In reply to Jeff Muizelaar [:jrmuizel] from comment #26)

Given the known major performance impact and no known privacy benefits, my suggestion is that we take this code out completely. Thoughts?

I agree that we can deprecate tailing tracker requests due to the reason you mentioned. But we cannot remove tailing entirely because I think it is still helpful in other cases.

Valentin, could you chime in here?

Flags: needinfo?(tihuang) → needinfo?(valentin.gosu)

(In reply to Tim Huang[:timhuang] from comment #28)

I agree that we can deprecate tailing tracker requests due to the reason you mentioned. But we cannot remove tailing entirely because I think it is still helpful in other cases.

What are the other cases?

Flags: needinfo?(tihuang)

Currently in Firefox, regular browsing windows, we tail third party tracking scripts
Tailing documentation

Here's a snippet of what we currently tail on cnn.com

nsHttpChannel::Connect -- set tracking resource https://pubads.g.doubleclick.net/pagead/interaction/...
nsHttpChannel::Connect -- set tracking resource https://bea4.v.fwmrm.net/ad/l/1?...
nsHttpChannel::Connect -- set tracking resource https://pixel.adsafeprotected.com/jload?anId=928791&pubId=16587&campId=preroll&pubOrder=88655447&pubCreative=278453098&placementId=88664033&chanId=993627&custom=&custom2= 
nsHttpChannel::Connect -- set tracking resource https://jsconfig.adsafeprotected.com/jsconfig/jload/928791?...
nsHttpChannel::Connect -- set tracking resource https://static.adsafeprotected.com/sca.17.6.4.js 
nsHttpChannel::Connect -- set tracking resource https://jsconfig.adsafeprotected.com/mon?cbName=__IntegralAS__2755&anId=928791&pubId=16587&campId=preroll&pubOrder=88655447&pubCreative=278453098&placementId=8866403

I don't have comparative evidence but, performance-wise, this is likely quite beneficial.
(I do have older evidence that strict tracking protection settings in which these requests are aborted instead of tailed is beneficial to navigation).

But these are different resources from the content tracking resources that we've fixed in this bug.

e.g. fonts, JS libraries, CSS.

nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/webfonts/fa-regular-400.woff2?_cb=17678861062740.45229169029494876 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/webfonts/fa-brands-400.woff2?_cb=17678861064750.6739377548159451 
nsHttpChannel::Connect -- set tracking resource https://fonts.gstatic.com/s/roboto/v30/KFOmCnqEu92Fr1Mu4mxP.ttf?_cb=17678861065810.9061266080781591 
nsHttpChannel::Connect -- set tracking resource https://fonts.gstatic.com/s/roboto/v30/KFOlCnqEu92Fr1MmWUlfBBc9.ttf?_cb=17678861069940.9105588216396399 
nsHttpChannel::Connect -- set tracking resource https://fonts.gstatic.com/s/lato/v24/S6uyw4BMUTPHjx4wWw.ttf?_cb=17678861070890.9548425476404192 
nsHttpChannel::Connect -- set tracking resource https://fonts.gstatic.com/s/montserrat/v26/JTUHjIg1_i6t8kCHKm4532VJOt5-QNFgpCtr6Ew-.ttf?_cb=17678861070900.9973151069889254 
nsHttpChannel::Connect -- set tracking resource https://fonts.gstatic.com/s/poppins/v20/pxiEyp8kv8JHgFVrFJA.ttf?_cb=17678861071280.21585601608560012 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/react/18.2.0/umd/react.production.min.js?_cb=17678861071350.48168178854789023 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/react-dom/18.2.0/umd/react-dom.production.min.js?_cb=17678861071350.8647942464024754 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/vue/3.4.21/vue.global.prod.min.js?_cb=17678861071360.16345545889949797 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.8.3/angular.min.js?_cb=17678861071360.22669395417756277 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.3.3/css/bootstrap.min.css?_cb=17678861086910.007818842398509829 
nsHttpChannel::Connect -- set tracking resource https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.2.19/tailwind.min.css?_cb=17678861086930.5536426384390355

(In reply to Jeff Muizelaar [:jrmuizel] from comment #29)

What are the other cases?

Valentin knows better than I do for other cases. I've ni'ed him for his insight.

Flags: needinfo?(tihuang)

Thanks to Andrew for adding the context in comment 30.
Indeed tailing for tracking resources like adpixels does usually show significant page load improvements (the exception being when the page hides content until tracking pixels are loaded).

For CDN "content tracking" we probably should not be tailing the resources at all as that's likely to hurt the page load - the scripts and css actually being required for the page to work.

Flags: needinfo?(valentin.gosu)

Ben, ni'ing you to request uplift for the dot release as I'll be out the rest of this week.

To me this is release note worthy (although the note could wait until Fx 148?)
Either way, I propose something simple along these lines:

Many popular websites now load faster, with progress bars completing sooner and content appearing more quickly.
Flags: needinfo?(bvandersloot)
See Also: → 2010113

Bug 2010113 looks at tailing content blockers in Private Browsing Mode.

QA Whiteboard: [qa-triage-done-c149/b148]

Uplift request made

Flags: needinfo?(bvandersloot)

firefox-release Uplift Approval Request

  • User impact if declined: Significant performance improvement available for Android users
  • Code covered by automated testing: yes
  • Fix verified in Nightly: yes
  • Needs manual QE test: no
  • Steps to reproduce for manual QE testing:
  • Risk associated with taking this patch: low
  • Explanation of risk level: Mostly just disabling a feature for android users. Currently working in Beta and Nightly
  • String changes made/needed: no
  • Is Android affected?: yes
Attachment #9537480 - Flags: approval-mozilla-release?
See Also: → 1915866
Attachment #9537480 - Flags: approval-mozilla-release? → approval-mozilla-release+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: