1794130 - Tracking protection list not downloaded on start up

Status: NEW

(GeckoView :: General, defect, P3)

Description

[Arturo Mejia [:amejia]]

We need to investigate why the tracking protection list is not download just after startup, as we are seeing long delays until the list is available.

Related tickets

Fenix
https://github.com/mozilla-mobile/fenix/issues/7907
Focus
https://github.com/mozilla-mobile/focus-android/issues/7821

Comment 1

[Chris Peterson [:cpeterson]]

Are Fenix and Focus UI tests allowed to use the network? Intermittent network errors or perf differences might prevent the ETP lists from being downloaded soon enough.

Firefox desktop tests aren't allowed to use the network. (If they do, they will crash.) This approach is to make the tests more reliable and deterministic. Tests must use a mock server running on localhost, though that might not be feasible on Android.

Comment 2

[Roger Yang [:royang]]

Using testing and console debugging, I've narrowed down the issue:

1. Fails to download tracking protection block list on first start.
2. Retry timer set to 20 minutes from first start.
3a. If the user doesn't restart Fenix, when retry timer expires, list will be downloaded. (100% success rate)
3b. If the user restart Fenix (swipe away first), then the list will be downloaded right away. (100% success rate)

In my opinion this issue is seen because of three reasons:

1. Inconsistent success rate when downloading block list at first start.
2. We don't have a default list built in.
3. Long backoff retry timer when we fails to download at first start.

Comment 3

[Chris Peterson [:cpeterson]]

I filed bug 1795481 about bundling a preloaded ETP list, but I don't know when it might be prioritized.

A new user temporarily browsing with an empty ETP list is not a fatal problem (especially after we ship TCP). So conclusively fixing this problem by preloading an initial ETP list is probably not a high priority. But tweaking our retry timer in the meantime is probably worthwhile.

Comment 4

[Dimi Lee [:dimi]]

(In reply to Chris Peterson [:cpeterson] from comment #3)

A new user temporarily browsing with an empty ETP list is not a fatal problem (especially after we ship TCP). So conclusively fixing this problem by preloading an initial ETP list is probably not a high priority. But tweaking our retry timer in the meantime is probably worthwhile.

Unfortunately, currently, the backoff timer is not configurable. This is because Safe Browsing spec defines the backoff algorithm, and we want to make sure we follow that.

Although it is still possible to make the backoff timer configurable only when list provider is "mozilla", I would prefer doing this only if we can't wait for fixing this issue with the "preloaded list" approach.

Comment 5

[Chris Peterson [:cpeterson]]

Tim Huang says Gecko doesn't support initializing the Shavar-based ETP list with local data, but Remote Settings does support initial data. Fixing this bug would require moving the ETP list from Shavar to Remote Settings.

https://firefox-source-docs.mozilla.org/services/settings/index.html#initial-data

This bug is not a high priority for now because the user exposure is small and we don't have an ETA for moving ETP to Remote Settings.

Comment 6

[Dan Mehic]

Bug 1795481 ("Bundle preloaded ETP list with Firefox install") was filed as a possible fix for this bug and has just been closed as WONTFIX — the approach was never adopted and its premise no longer matches the tree. Copying the current state of the world here, since this is where the actual defect lives and it is still open.

The ETP list now comes from Remote Settings, not Shavar

Bug 1795481 assumed a Shavar-to-RemoteSettings migration was a prerequisite. That has shipped. modules/libpref/init/all.js#3447:

pref("browser.safebrowsing.provider.mozilla.updateURL", "moz-sbrs:://antitracking");

nsUrlClassifierStreamUpdater::FetchUpdate routes the moz-sbrs scheme to UrlClassifierRemoteSettingsService, which reads the tracking-protection-lists collection. There is no override of that pref anywhere in the tree.

So anything here that was previously diagnosed in terms of Shavar fetch behaviour should be re-checked against the Remote Settings path instead.

The startup bundle is not a fix for this

Searching for tracking-protection-lists turns up a Remote Settings "startup bundle" that names this collection, which reads like it would solve the problem. It does not. Utils.fetchChangesetsBundle (Utils.sys.mjs#640) downloads bundles/startup.json.mozlz4 from the CDN. It collapses many per-collection requests into one, which helps startup latency, but it is still a network fetch and does nothing when the network is unavailable or flaky at first run.

Nothing is packaged in the install

• No tracking-protection-lists.json exists under services/settings/dumps/ in any bucket, and dumps/main/moz.build does not package one.
• Asserted directly by test_remote_settings_startup_bundle.js#67: Assert.ok(!(await Utils.hasLocalDump(...)), "Client has no packaged dump").

Two Android-specific notes

1. dumps/main/moz.build excludes mobile by design — the comment there reads "Android/iOS don't use/want the dumps". Any preload-style fix for GeckoView would have to revisit that exclusion first, not just add a dump file.
2. The list payload is stored in Remote Settings attachments, not in the records themselves — rs.attachments.downloadAsBytes(entry) at UrlClassifierRemoteSettingsService.sys.mjs#99. So a records-only dump would not give you a usable list. Packaged attachments are supported now, though: search-config-icons ships around 50 of them through FINAL_TARGET_FILES.defaults.settings.main["search-config-icons"] in the same moz.build.

Scope of the exposure, for prioritisation

On desktop, network.cookie.cookieBehavior defaults to 5 (dFPI), which partitions third-party cookies regardless of the tracking list, so the cookie vector is covered even with no list. But privacy.trackingprotection.fingerprinting.enabled and privacy.trackingprotection.cryptomining.enabled are both true by default and do consult the list, so those protections are genuinely absent until the first successful sync. GeckoView configures ETP through the ContentBlocking API at runtime rather than through those prefs, so the practical impact on Fenix should be assessed separately — that is really the open question on this bug.

No action requested here, just consolidating the findings so they are not lost with bug 1795481.